{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# 第一章~第二章：机器学习简介及线性回归...............学生：侯松林\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 基础作业"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 对连续型特征，可以用哪个函数可视化其分布？（给出你最常用的一个即可），并根据代码运行结果给出示例。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "答：matplotlib中的hist函数方法，可以画出数据的直方图，示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\envs\\tf1.14\\lib\\site-packages\\ipykernel_launcher.py:5: MatplotlibDeprecationWarning: \n",
      "The 'normed' kwarg was deprecated in Matplotlib 2.1 and will be removed in 3.1. Use 'density' instead.\n",
      "  \"\"\"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAU7klEQVR4nO3dcZAe933X8fcHqXKbBjutfDCJJOZUrFDOnaFOb4QLaQePSyyHNjLFhfNQasAzGhgbEkoGZDLJZEz/QNDUJWMnHRGLum4ayaOm0xtwUaEWMJkhsh87IYksq7nKGXS1SQ7sUeIERz33yx/PKnny+Lm71el098j7fs3cePe3v93nu49X99nf7vPspaqQJHXPn9joAiRJG8MAkKSOMgAkqaMMAEnqKANAkjpq80YXcDGuvfbampyc3OgyJOmKce2113Ls2LFjVbVneNkVFQCTk5P0er2NLkOSrihJrh3V7iUgSeooA0CSOsoAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6qgr6pvA0uvRTTdt3GsfP75xr62N5whAkjrKAJCkjvISkNS4lEsxXkrRlcgRgCR1lCMAaQ1s5I1cabUcAUhSRxkAktRRBoAkdZQBIEkd1SoAkuxJcjrJXJL9I5ZfleRIs/xEksmmfWuS40leTvLAEtueTfKFS9kJSdLFWzEAkmwCHgRuBaaAO5JMDXW7C3ipqq4D7gcONO2vAO8H3rvEtn8aeHl1pUuSLkWbEcBuYK6qzlTVeeAwsHeoz17g4Wb6KHBzklTV16vqU/SD4DskeSPw88AvrLp6SdKqtQmAbcDZgfn5pm1kn6paBM4BW1fY7r8EPgR8Y7lOSfYl6SXpLSwstChXktRGmwDIiLZaRZ9vd05+GLiuqn5rpRevqoNVNV1V0xMTEyt1lyS11CYA5oEdA/PbgeeX6pNkM3AN8OIy2/xR4EeSfAn4FPDWJP+1XcmSpLXQJgCeBHYl2ZlkCzADzA71mQXubKZvBx6vqiVHAFX10ap6S1VNAm8Hfr+q/srFFi9JWr0VnwVUVYtJ7gGOAZuAQ1V1Msl9QK+qZoGHgEeSzNE/85+5sH5zln81sCXJbcA7quqZtd8VSdLFyDIn6mNnenq6er3eRpeh1ykf6HZxfAT2lSPJU1U1PdzuN4ElqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSpowwASeooA0CSOsoAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6qlUAJNmT5HSSuST7Ryy/KsmRZvmJJJNN+9Ykx5O8nOSBgf5vSPIfkzyb5GSSf7VWOyRJamfFAEiyCXgQuBWYAu5IMjXU7S7gpaq6DrgfONC0vwK8H3jviE3/YlX9IHAD8JeT3Lq6XZAkrUabEcBuYK6qzlTVeeAwsHeoz17g4Wb6KHBzklTV16vqU/SD4Fuq6htVdbyZPg88DWy/hP2QJF2kNgGwDTg7MD/ftI3sU1WLwDlga5sCkrwJ+Cng95ZYvi9JL0lvYWGhzSYlSS20CYCMaKtV9HnthpPNwCeAD1fVmVF9qupgVU1X1fTExMSKxUqS2mkTAPPAjoH57cDzS/VpfqlfA7zYYtsHgS9W1S+36CtJWkNtAuBJYFeSnUm2ADPA7FCfWeDOZvp24PGqWnYEkOQX6AfFey6uZEnSWti8UoeqWkxyD3AM2AQcqqqTSe4DelU1CzwEPJJkjv6Z/8yF9ZN8Cbga2JLkNuAdwFeB9wHPAk8nAXigqj62ljsnSVraigEAUFWPAY8NtX1gYPoV4GeWWHdyic2Oum8gSVonfhNYkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSpowwASeooA0CSOsoAkKSOMgAkqaNaPQtIulLcdNNGVyBdORwBSFJHGQCS1FEGgCR1lAEgSR1lAEhSRxkAktRRrQIgyZ4kp5PMJdk/YvlVSY40y08kmWzatyY5nuTlJA8MrfMjST7frPPhNH8YWJK0PlYMgCSbgAeBW4Ep4I4kU0Pd7gJeqqrrgPuBA037K8D7gfeO2PRHgX3AruZnz2p2QJK0Om1GALuBuao6U1XngcPA3qE+e4GHm+mjwM1JUlVfr6pP0Q+Cb0nyZuDqqvofVVXArwG3XcqOSJIuTpsA2AacHZifb9pG9qmqReAcsHWFbc6vsE1J0mXUJgBGXZuvVfRZVf8k+5L0kvQWFhaW2aQk6WK0CYB5YMfA/Hbg+aX6JNkMXAO8uMI2t6+wTQCq6mBVTVfV9MTERItyJUlttAmAJ4FdSXYm2QLMALNDfWaBO5vp24HHm2v7I1XVC8DXktzYfPrn54DfvujqJUmrtuLTQKtqMck9wDFgE3Coqk4muQ/oVdUs8BDwSJI5+mf+MxfWT/Il4GpgS5LbgHdU1TPAPwR+Ffge4HeaH0nSOmn1OOiqegx4bKjtAwPTrwA/s8S6k0u094AfaluoJGlt+U1gSeooA0CSOsoAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSpowwASeooA0CSOsoAkKSOMgAkqaMMAEnqKANAkjrKAJCkjmoVAEn2JDmdZC7J/hHLr0pypFl+IsnkwLJ7m/bTSW4ZaP8nSU4m+UKSTyT57rXYIUlSOysGQJJNwIPArcAUcEeSqaFudwEvVdV1wP3AgWbdKWAGuB7YA3wkyaYk24B/DExX1Q8Bm5p+kqR10mYEsBuYq6ozVXUeOAzsHeqzF3i4mT4K3JwkTfvhqvpmVT0HzDXbA9gMfE+SzcAbgOcvbVckSRejTQBsA84OzM83bSP7VNUicA7YutS6VfWHwC8C/wt4AThXVb876sWT7EvSS9JbWFhoUa4kqY02AZARbdWyz8j2JN9Hf3SwE3gL8L1JfnbUi1fVwaqarqrpiYmJFuVKktpoEwDzwI6B+e289nLNt/o0l3SuAV5cZt2fAJ6rqoWq+iPgk8BfWs0OSJJWp00APAnsSrIzyRb6N2tnh/rMAnc207cDj1dVNe0zzaeEdgK7gCfoX/q5MckbmnsFNwOnLn13JEltbV6pQ1UtJrkHOEb/0zqHqupkkvuAXlXNAg8BjySZo3/mP9OsezLJo8AzwCJwd1W9CpxIchR4umn/DHBw7XdPkrSU9E/UrwzT09PV6/U2ugyNsZtu2ugKuuP48Y2uQG0leaqqpofb/SawJHWUASBJHWUASFJHGQCS1FEGgCR1lAEgSR214vcApPXkxzil9eMIQJI6ygCQpI4yACSpo7wHIGlVLuV+jY+RGA+OACSpowwASeooA0CSOsoAkKSOMgAkqaMMAEnqKANAkjrKAJCkjmoVAEn2JDmdZC7J/hHLr0pypFl+IsnkwLJ7m/bTSW4ZaH9TkqNJnk1yKsmPrsUOSZLaWTEAkmwCHgRuBaaAO5JMDXW7C3ipqq4D7gcONOtOATPA9cAe4CPN9gD+LfCfquoHgb8AnLr03ZEktdXmURC7gbmqOgOQ5DCwF3hmoM9e4IPN9FHggSRp2g9X1TeB55LMAbuTnAR+HPi7AFV1Hjh/yXsj6YrgYyTGQ5tLQNuAswPz803byD5VtQicA7Yus+4PAAvAv0/ymSQfS/K9o148yb4kvSS9hYWFFuVKktpoEwAZ0VYt+yzVvhl4G/DRqroB+DrwmnsLAFV1sKqmq2p6YmKiRbmSpDbaBMA8sGNgfjvw/FJ9kmwGrgFeXGbdeWC+qk407UfpB4IkaZ20CYAngV1JdibZQv+m7uxQn1ngzmb6duDxqqqmfab5lNBOYBfwRFX9b+Bskj/XrHMz33lPQZJ0ma14E7iqFpPcAxwDNgGHqupkkvuAXlXNAg8BjzQ3eV+kHxI0/R6l/8t9Ebi7ql5tNv2PgI83oXIG+HtrvG+SpGWkf6J+ZZienq5er7fRZegy8o/CayV+CujiJXmqqqaH2/0msCR1lAEgSR1lAEhSRxkAktRRBoAkdZQBIEkdZQBIUkcZAJLUUW0eBy1dFL/MJV0ZHAFIUkcZAJLUUQaAJHWUASBJHWUASFJHGQCS1FEGgCR1lAEgSR1lAEhSR7UKgCR7kpxOMpdk/4jlVyU50iw/kWRyYNm9TfvpJLcMrbcpyWeS/IdL3RFJ0sVZMQCSbAIeBG4FpoA7kkwNdbsLeKmqrgPuBw40607R/wPx1wN7gI8027vg3cCpS90JSdLFazMC2A3MVdWZqjoPHAb2DvXZCzzcTB8Fbk6Spv1wVX2zqp4D5prtkWQ78NeAj136bkiSLlabANgGnB2Yn2/aRvapqkXgHLB1hXV/GfhnwB8v9+JJ9iXpJektLCy0KFeS1EabAMiItmrZZ2R7kp8EvlJVT6304lV1sKqmq2p6YmJi5WolSa20eRz0PLBjYH478PwSfeaTbAauAV5cZt13Ae9K8k7gu4Grk/x6Vf3sqvZCUmdc6uPGjx9fmzpeD9qMAJ4EdiXZmWQL/Zu6s0N9ZoE7m+nbgcerqpr2meZTQjuBXcATVXVvVW2vqslme4/7y1+S1teKI4CqWkxyD3AM2AQcqqqTSe4DelU1CzwEPJJkjv6Z/0yz7skkjwLPAIvA3VX16mXaF0nSRUj/RP3KMD09Xb1eb6PL0Ar8i2AaZ128BJTkqaqaHm73m8CS1FEGgCR1lAEgSR1lAEhSRxkAktRRBoAkdZQBIEkdZQBIUkcZAJLUUQaAJHVUm6eBqoN8nIP0+ucIQJI6ygCQpI4yACSpowwASeooA0CSOsoAkKSOMgAkqaMMAEnqqFYBkGRPktNJ5pLsH7H8qiRHmuUnkkwOLLu3aT+d5JambUeS40lOJTmZ5N1rtUOSpHZWDIAkm4AHgVuBKeCOJFND3e4CXqqq64D7gQPNulPADHA9sAf4SLO9ReCfVtWfB24E7h6xTUnSZdRmBLAbmKuqM1V1HjgM7B3qsxd4uJk+CtycJE374ar6ZlU9B8wBu6vqhap6GqCqvgacArZd+u5IktpqEwDbgLMD8/O89pf1t/pU1SJwDtjaZt3mctENwIlRL55kX5Jekt7CwkKLciVJbbQJgIxoq5Z9ll03yRuB3wTeU1VfHfXiVXWwqqaranpiYqJFuZKkNtoEwDywY2B+O/D8Un2SbAauAV5cbt0k30X/l//Hq+qTqylekrR6bQLgSWBXkp1JttC/qTs71GcWuLOZvh14vKqqaZ9pPiW0E9gFPNHcH3gIOFVVv7QWOyJJujgr/j2AqlpMcg9wDNgEHKqqk0nuA3pVNUv/l/kjSebon/nPNOueTPIo8Az9T/7cXVWvJnk78HeAzyf5bPNS/6KqHlvrHZQkjZb+ifqVYXp6unq93kaX0Qn+QRi9Xh0/vtEVrL8kT1XV9HC73wSWpI4yACSpo/ybwGPuUi7FdHGoK6k9RwCS1FEGgCR1lJeAXsf8JI/0Wl5W/TZHAJLUUQaAJHWUASBJHWUASFJHGQCS1FEGgCR1lAEgSR3l9wAkqaXX23cIHAFIUkc5ApCkdTCOowdHAJLUUQaAJHWUl4AuMx/IJmlctRoBJNmT5HSSuST7Ryy/KsmRZvmJJJMDy+5t2k8nuaXtNiVJl9eKI4Akm4AHgb8KzANPJpmtqmcGut0FvFRV1yWZAQ4AfyvJFDADXA+8BfgvSd7arLPSNtfUON6AkaSN1GYEsBuYq6ozVXUeOAzsHeqzF3i4mT4K3JwkTfvhqvpmVT0HzDXba7NNSdJl1OYewDbg7MD8PPAXl+pTVYtJzgFbm/ZPD627rZleaZsAJNkH7GtmX05yukXNayq5pNWvBf7P2lSyrqx7fVn3+rqi6h74HbSaupfs3yYARv36q5Z9lmofNfIY3ma/seogcHC5AsdZkl5VTW90HRfLuteXda8v6+5rcwloHtgxML8deH6pPkk2A9cALy6zbpttSpIuozYB8CSwK8nOJFvo39SdHeozC9zZTN8OPF5V1bTPNJ8S2gnsAp5ouU1J0mW04iWg5pr+PcAxYBNwqKpOJrkP6FXVLPAQ8EiSOfpn/jPNuieTPAo8AywCd1fVqwCjtrn2uzcWrtTLV9a9vqx7fVk3kP6JuiSpa3wUhCR1lAEgSR1lAFyCJIeSfCXJFwba/k2SZ5N8LslvJXlT0z6Z5P8l+Wzz8ytjVvcHk/zhQH3vHFg28nEe622Juo8M1PylJJ9t2sfp/d6R5HiSU0lOJnl30/79Sf5zki82//2+pj1JPty8559L8rYxq3usj/Fl6h7rY3yZui/fMV5V/qzyB/hx4G3AFwba3gFsbqYPAAea6cnBfmNY9weB947oOwX8T+AqYCfwB8Cmcal7aPmHgA+M4fv9ZuBtzfSfBH6/eV//NbC/ad8/cKy8E/gd+t+juRE4MWZ1j/UxvkzdY32ML1X3UJ81PcYdAVyCqvrv9D/1NNj2u1W12Mx+mv53HMbKqLqXsdTjPNbdcnU3jx75m8An1rWoFqrqhap6upn+GnCK/jfiBx+h8jBwWzO9F/i16vs08KYkb17nspese9yP8WXe76WMxTG+Ut2X4xg3AC6vv0//TO6CnUk+k+S/JfmxjSpqGfc0w/pDFy5HMPpRIMv9Y9ooPwZ8uaq+ONA2du93+k/KvQE4AfzpqnoB+v/4gT/VdBu793yo7kFjfYyPqPuKOMaXeL/X/Bg3AC6TJO+j/92HjzdNLwB/pqpuAH4e+I0kV29UfSN8FPizwA/Tr/VDTXubR4GMgzv4zjOjsXu/k7wR+E3gPVX11eW6jmjbsPd8qbrH/RgfUfcVcYwvc5ys+TFuAFwGSe4EfhL429VcrGuGl/+3mX6K/nXGty69lfVVVV+uqler6o+Bf8e3h8Bj/9iO9B8/8tPAkQtt4/Z+J/ku+v+oP15Vn2yav3zh0k7z36807WPzni9R99gf46PqvhKO8WXe78tyjBsAayzJHuCfA++qqm8MtE+k/7cVSPID9B+LcWZjqnytoWvMfx248EmbpR7nMU5+Ani2quYvNIzT+91cu30IOFVVvzSwaPARKncCvz3Q/nPNp4FuBM5duFS0npaqe9yP8WXqHutjfJnjBC7XMb7ed7pfTz/0h2MvAH9E/yziLvo3kM4Cn21+fqXp+zeAk/Q/bfA08FNjVvcjwOeBz9H/B/Hmgf7vo392cRq4dZzqbtp/FfgHQ33H6f1+O/1LCp8bOC7eSf+R6b8HfLH57/c3/UP/Dyb9QfP/ZHrM6h7rY3yZusf6GF+q7st5jPsoCEnqKC8BSVJHGQCS1FEGgCR1lAEgSR1lAEhSRxkAktRRBoAkddT/BwdRHTlCkYsNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "mu,sigma = 200,25 #均值与标准差\n",
    "x = mu+sigma*np.random.randn(1000) #产生数据\n",
    "plt.hist(x,20,normed=1,histtype='stepfilled',facecolor='b',alpha=0.75) #画出直方图\n",
    "plt.show() #展示结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 对两个连续型特征，可以用哪个函数得到这两个特征之间的相关性？根据代码运行结果，给出示例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "答：DataFrame中的corr()，可以计算各特征间的相关性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             CRIM        ZN     INDUS      CHAS       NOX        RM       AGE  \\\n",
      "CRIM     1.000000 -0.200294  0.406583 -0.055892  0.420972 -0.219247  0.352734   \n",
      "ZN      -0.200294  1.000000 -0.533583 -0.042533 -0.516310  0.312218 -0.569483   \n",
      "INDUS    0.406583 -0.533583  1.000000  0.062938  0.763651 -0.391676  0.644779   \n",
      "CHAS    -0.055892 -0.042533  0.062938  1.000000  0.091203  0.091251  0.086518   \n",
      "NOX      0.420972 -0.516310  0.763651  0.091203  1.000000 -0.302188  0.731470   \n",
      "RM      -0.219247  0.312218 -0.391676  0.091251 -0.302188  1.000000 -0.240265   \n",
      "AGE      0.352734 -0.569483  0.644779  0.086518  0.731470 -0.240265  1.000000   \n",
      "DIS     -0.379670  0.663845 -0.708027 -0.099176 -0.769230  0.205246 -0.747881   \n",
      "RAD      0.625505 -0.311712  0.595129 -0.007368  0.611441 -0.209847  0.456022   \n",
      "TAX      0.582764 -0.314338  0.720760 -0.035587  0.668023 -0.292048  0.506456   \n",
      "PTRATIO  0.303369 -0.377985  0.398443 -0.129580  0.208947 -0.347275  0.274111   \n",
      "B       -0.385064  0.175341 -0.356977  0.048788 -0.380051  0.128069 -0.273534   \n",
      "LSTAT    0.455621 -0.413195  0.603800 -0.053929  0.590879 -0.613808  0.602339   \n",
      "MEDV    -0.388305  0.360580 -0.483725  0.175260 -0.427321  0.695360 -0.376955   \n",
      "\n",
      "              DIS       RAD       TAX   PTRATIO         B     LSTAT      MEDV  \n",
      "CRIM    -0.379670  0.625505  0.582764  0.303369 -0.385064  0.455621 -0.388305  \n",
      "ZN       0.663845 -0.311712 -0.314338 -0.377985  0.175341 -0.413195  0.360580  \n",
      "INDUS   -0.708027  0.595129  0.720760  0.398443 -0.356977  0.603800 -0.483725  \n",
      "CHAS    -0.099176 -0.007368 -0.035587 -0.129580  0.048788 -0.053929  0.175260  \n",
      "NOX     -0.769230  0.611441  0.668023  0.208947 -0.380051  0.590879 -0.427321  \n",
      "RM       0.205246 -0.209847 -0.292048 -0.347275  0.128069 -0.613808  0.695360  \n",
      "AGE     -0.747881  0.456022  0.506456  0.274111 -0.273534  0.602339 -0.376955  \n",
      "DIS      1.000000 -0.494588 -0.534432 -0.241996  0.291512 -0.496996  0.249929  \n",
      "RAD     -0.494588  1.000000  0.910228  0.485530 -0.444413  0.488676 -0.381626  \n",
      "TAX     -0.534432  0.910228  1.000000  0.478160 -0.441808  0.543993 -0.468536  \n",
      "PTRATIO -0.241996  0.485530  0.478160  1.000000 -0.184927  0.383282 -0.506398  \n",
      "B        0.291512 -0.444413 -0.441808 -0.184927  1.000000 -0.366087  0.333461  \n",
      "LSTAT   -0.496996  0.488676  0.543993  0.383282 -0.366087  1.000000 -0.737663  \n",
      "MEDV     0.249929 -0.381626 -0.468536 -0.506398  0.333461 -0.737663  1.000000  \n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_boston\n",
    "df = pd.read_csv(\"boston_housing.csv\") #导入数据\n",
    "data_corr=df.corr() #计算各列相关系\n",
    "print(data_corr) #打印出结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.如果发现特征之间有较强的相关性，在选择线性回归模型时应该采取什么措施。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "答：如果特征间高度相关，可考虑进行PCA降维（特征层面）或加正则项（模型层面）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.当采用带正则的模型以及采用随机梯度下降优化算法时，需要对输入（连续型）特征进行去量纲预处理。课程代码给出了用标准化（StandardScaler）的结果，请改成最小最大缩放（MinMaxScaler）去量纲 ，并重新训练最小二乘线性回归、岭回归、和Lasso模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1)先将数据进行MinMaxScaler去量纲，并保存到mm_boston_housing.csv文件中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_boston\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "df = pd.read_csv(\"boston_housing.csv\") #导入数据\n",
    "y=df['MEDV'] #取出MEDV特征数据\n",
    "x=df.drop('MEDV',axis=1) #除去MEDV列特征数据\n",
    "log_y = np.log1p(y) #计算MEDV特征的log值\n",
    "x['RAD'].astype(\"object\") #将类别型特征进行编码转化\n",
    "x_cat = x['RAD']\n",
    "x_cat = pd.get_dummies(x_cat,prefix='RAD')\n",
    "x = x.drop('RAD',axis=1)\n",
    "feat_names = x.columns\n",
    "ss_x = MinMaxScaler() #最小最大缩放（MinMaxScaler）去量纲\n",
    "ss_y = MinMaxScaler()#最小最大缩放（MinMaxScaler）去量纲\n",
    "ss_log_y = MinMaxScaler() #最小最大缩放（MinMaxScaler）去量纲\n",
    "x = ss_x.fit_transform(x)\n",
    "y = ss_y.fit_transform(y.values.reshape(-1,1))\n",
    "log_y = ss_y.fit_transform(log_y.values.reshape(-1,1))\n",
    "fe_data = pd.DataFrame(data=x,columns=feat_names,index=df.index)\n",
    "fe_data = pd.concat([fe_data,x_cat],axis=1,ignore_index=False)\n",
    "fe_data['MEDV'] = y\n",
    "fe_data['log_MEDV'] = log_y\n",
    "fe_data.to_csv('mm_boston_housing.csv',index=False) #将预处理后的数据进行保存"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2)训练最小二乘回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of LinearRegression on test is 0.6939789810509469\n",
      "The r2 score of LinearRegression on train is 0.7549146436868177\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(array([ 1.,  0.,  0.,  4., 10., 16., 18., 21., 31., 47., 38., 45., 46.,\n",
       "        30., 25., 16., 14.,  5.,  9.,  4.,  2.,  2.,  3.,  2.,  3.,  5.,\n",
       "         0.,  2.,  0.,  0.,  1.,  1.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,\n",
       "         2.]),\n",
       " array([-2.40385685e-01, -2.20319301e-01, -2.00252918e-01, -1.80186535e-01,\n",
       "        -1.60120151e-01, -1.40053768e-01, -1.19987384e-01, -9.99210010e-02,\n",
       "        -7.98546176e-02, -5.97882342e-02, -3.97218508e-02, -1.96554674e-02,\n",
       "         4.10916055e-04,  2.04772995e-02,  4.05436829e-02,  6.06100663e-02,\n",
       "         8.06764497e-02,  1.00742833e-01,  1.20809217e-01,  1.40875600e-01,\n",
       "         1.60941983e-01,  1.81008367e-01,  2.01074750e-01,  2.21141134e-01,\n",
       "         2.41207517e-01,  2.61273900e-01,  2.81340284e-01,  3.01406667e-01,\n",
       "         3.21473051e-01,  3.41539434e-01,  3.61605817e-01,  3.81672201e-01,\n",
       "         4.01738584e-01,  4.21804968e-01,  4.41871351e-01,  4.61937734e-01,\n",
       "         4.82004118e-01,  5.02070501e-01,  5.22136885e-01,  5.42203268e-01,\n",
       "         5.62269652e-01]),\n",
       " <a list of 40 Patch objects>)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAFcCAYAAAAK4I0VAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAQbklEQVR4nO3de4yld13H8c+XLogi2pZuofbiQixIJVqStZKQiLYQq2hpYtWikCUpblCIGEy0Cn+sl0TqH4CJjVqBuBilhXpprREDpQ1gaGELBSwNbFtRmzbtcingDan8/GOehnHZ7Zy5nDnznXm9ks2cc+Y5M9/fnp19z3PmOc/UGCMAwNb2mEUPAACsTLABoAHBBoAGBBsAGhBsAGhg12Z+slNOOWXs2bNnMz8lAGwpt91222fHGLtXe79NDfaePXty6NChzfyUALClVNW/rOV+nhIHgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoQLABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAa2NRfr0l/Bw5s7HYAzMYeNgA0INgA0IBgA0ADgg0ADQg2ADTgKHHacIQ6sJPZwwaABgQbABoQbABoQLABoAHBBoAGHCXOQjmiG2A29rABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoQLABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoQLABoAHBBoAGBBsAGhBsAGhg5mBX1QlV9dGqumG6/tSqurWqDlfVNVX1uPmNCQA722r2sF+d5M5l169I8sYxxtlJvpDkso0cDAD4upmCXVVnJHlhkjdP1yvJ+UmunTY5mOTieQwIAMy+h/2mJL+a5GvT9ScleWiM8fB0/d4kp2/wbADAZMVgV9WPJ3lwjHHb8puPsek4zv33V9Whqjp05MiRNY4JADvbLHvYz01yUVV9JsnVWXoq/E1JTqyqXdM2ZyS571h3HmNcNcbYO8bYu3v37g0YGQB2nhWDPcb49THGGWOMPUkuTfLeMcbPJbkpySXTZvuSXDe3KQFgh1vP67B/LclrququLP1M+y0bMxIAcLRdK2/ydWOMm5PcPF2+J8l5Gz8SAHA0ZzoDgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoYFWvw4ZZHTiw6AkAthd72ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0sGvRA7B4Bw4segIAVmIPGwAaEGwAaECwAaCBFYNdVY+vqg9V1ceq6o6q+s3p9qdW1a1Vdbiqrqmqx81/XADYmWbZw/5KkvPHGN+X5NwkF1bVc5JckeSNY4yzk3whyWXzGxMAdrYVgz2W/Pt09bHTn5Hk/CTXTrcfTHLxXCYEAGZ7WVdVnZDktiTfleTKJHcneWiM8fC0yb1JTj/Offcn2Z8kZ5111nrnhRXN+jI1L2cDOpnpoLMxxv+OMc5NckaS85I881ibHee+V40x9o4x9u7evXvtkwLADraqo8THGA8luTnJc5KcWFWP7KGfkeS+jR0NAHjELEeJ766qE6fL35zk+UnuTHJTkkumzfYluW5eQwLATjfLz7BPS3Jw+jn2Y5K8Y4xxQ1V9MsnVVfU7ST6a5C1znBMAdrQVgz3G+HiSZx/j9nuy9PNsAGDOnOkMABoQbABoQLABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoQLABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoYNeiB4BFOXBgY7cDmCd72ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQwK5FD8D8HDiw6AkA2Cj2sAGgAcEGgAZWDHZVnVlVN1XVnVV1R1W9err95Kp6d1Udnt6eNP9xAWBnmmUP++EkvzLGeGaS5yR5ZVWdk+TyJDeOMc5OcuN0HQCYgxWDPca4f4zxkenyl5PcmeT0JC9KcnDa7GCSi+c1JADsdKv6GXZV7Uny7CS3JnnyGOP+ZCnqSU49zn32V9Whqjp05MiR9U0LADvUzMGuqm9N8pdJfnmM8aVZ7zfGuGqMsXeMsXf37t1rmREAdryZgl1Vj81SrP98jPFX080PVNVp0/tPS/LgfEYEAGY5SrySvCXJnWOMNyx71/VJ9k2X9yW5buPHAwCS2c509twkL03yiaq6fbrtN5K8Psk7quqyJP+a5KfmMyIAsGKwxxgfSFLHefcFGzsOAHAsznQGAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQwCwnTmGLOXBg0RMAsNnsYQNAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANCDYANCDYANCAX/6xhfilHlvTah4XjyEwL/awAaABwQaABgQbABoQbABoQLABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaMAv/4ANNOsv//BLQoDVsocNAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANCDYANCDYANCAYANAA4INAA0INgA0INgA0IBgA0ADgg0ADQg2ADQg2ADQgGADQAOCDQANrBjsqnprVT1YVf+07LaTq+rdVXV4envSfMcEgJ1tlj3sP01y4VG3XZ7kxjHG2UlunK4DAHOyYrDHGO9L8vmjbn5RkoPT5YNJLt7guQCAZXat8X5PHmPcnyRjjPur6tTjbVhV+5PsT5KzzjprjZ+urwMHFj0BANvB3A86G2NcNcbYO8bYu3v37nl/OgDYltYa7Aeq6rQkmd4+uHEjAQBHW2uwr0+yb7q8L8l1GzMOAHAss7ys6+1JPpjkGVV1b1VdluT1SV5QVYeTvGC6DgDMyYoHnY0xXnycd12wwbMAAMfhTGcA0MBaX9YFbIJZXxbo5YOw/dnDBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoQLABoAHBBoAGBBsAGhBsAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABwQaABgQbABoQbABoQLABoAHBBoAGdi16ANiJDhxY9ATbw6x/j/6+2Q7sYQNAA4INAA0INgA0INgA0IBgA0ADjhJfI0edspWs5t+jf7vQkz1sAGhAsAGgAcEGgAYEGwAaEGwAaECwAaABL+sC1mUev4DDS8/gG9nDBoAGBBsAGhBsAGhAsAGgAcEGgAYcJQ47zKKOwO5w5Lcj2XeWebzCYZ7sYQNAA4INAA0INgA0INgA0IBgA0ADrY8Sd0QnsF3M4/+o7fT/XrcjuufBHjYANCDYANCAYANAA+sKdlVdWFWfqqq7quryjRoKAPj/1hzsqjohyZVJfjTJOUleXFXnbNRgAMDXrWcP+7wkd40x7hlj/E+Sq5O8aGPGAgCWqzHG2u5YdUmSC8cYL5+uvzTJD4wxXnXUdvuT7J+uPiPJp9Y+7qY4JclnFz3EBrGWrWk7rSXZXuuxlq1pu63lCWOM3au943peh13HuO0b6j/GuCrJVev4PJuqqg6NMfYueo6NYC1b03ZaS7K91mMtW9M2XMuetdx3PU+J35vkzGXXz0hy3zo+HgBwHOsJ9oeTnF1VT62qxyW5NMn1GzMWALDcmp8SH2M8XFWvSvIPSU5I8tYxxh0bNtnitHn6fgbWsjVtp7Uk22s91rI1WUvWcdAZALB5nOkMABoQbABoYMcHu6pOrqp3V9Xh6e1Jx9jm3Kr6YFXdUVUfr6qfWcSsK5llLdN276qqh6rqhs2ecSUrne62qr6pqq6Z3n9rVe3Z/ClnM8NafrCqPlJVD0/nNdiyZljLa6rqk9PXx41V9Z2LmHMWM6zlFVX1iaq6vao+sNXP4DjrKaKr6pKqGlW1ZV8eNcNj87KqOjI9NrdX1csXMecsZnlcquqnp6+bO6rqL1b8oGOMHf0nye8luXy6fHmSK46xzdOTnD1d/o4k9yc5cdGzr2Ut0/suSPITSW5Y9MxHzXVCkruTPC3J45J8LMk5R23zi0n+aLp8aZJrFj33OtayJ8n3JnlbkksWPfM61/LDSb5luvwLzR+Xb1t2+aIk71r03OtZz7TdE5O8L8ktSfYueu51PDYvS/IHi551g9ZydpKPJjlpun7qSh93x+9hZ+l0qgenyweTXHz0BmOMT48xDk+X70vyYJJVn6VmE6y4liQZY9yY5MubNdQqzHK62+VrvDbJBVV1rJP4LNqKaxljfGaM8fEkX1vEgKswy1puGmP853T1liydl2ErmmUtX1p29Qk5xgmhtpBZTxH921n6hv6/N3O4VdpOp7ueZS0/n+TKMcYXkmSM8eBKH1SwkyePMe5PkuntqY+2cVWdl6XvmO7ehNlWa1Vr2YJOT/Jvy67fO912zG3GGA8n+WKSJ23KdKszy1q6WO1aLkvy93OdaO1mWktVvbKq7s5S5H5pk2ZbixXXU1XPTnLmGGPL/QjsKLP+O/vJ6Ucv11bVmcd4/1Ywy1qenuTpVfWPVXVLVV240gddz6lJ26iq9yR5yjHe9dpVfpzTkvxZkn1jjIXsFW3UWraoWU53O9MpcbeALnPOYua1VNVLkuxN8ry5TrR2s55S+cokV1bVzyZ5XZJ98x5sjR51PVX1mCRvzNJTyVvdLI/N3yZ5+xjjK1X1iiw923b+3CdbvVnWsitLT4v/UJaekXp/VT1rjPHQ8T7ojgj2GOP5x3tfVT1QVaeNMe6fgnzMpyWq6tuS/F2S140xbpnTqCvaiLVsYbOc7vaRbe6tql1Jvj3J5zdnvFXZTqfunWktVfX8LH3j+Lwxxlc2abbVWu3jcnWSP5zrROuz0nqemORZSW6efnL0lCTXV9VFY4xDmzblbFZ8bMYYn1t29U+SXLEJc63FrP+X3TLG+GqSf66qT2Up4B8+3gf1lPjS6VQf+e55X5Lrjt5gOvXqXyd52xjjnZs422qtuJYtbpbT3S5f4yVJ3jumIza2mO106t4V1zI97frHSS6a5WdxCzTLWs5edvWFSQ5v4nyr9ajrGWN8cYxxyhhjz1j6hRO3ZOkx2mqxTmZ7bE5bdvWiJHdu4nyrMcvX/99k6WDNVNUpWXqK/J5H/aiLPppu0X+y9PPPG7P0RXljkpOn2/cmefN0+SVJvprk9mV/zl307GtZy3T9/UmOJPmvLH2X9yOLnn3ZbD+W5NNZOkbgtdNtv5Wl/2SS5PFJ3pnkriQfSvK0Rc+8jrV8//T3/x9JPpfkjkXPvI61vCfJA8u+Pq5f9MzrWMvvJ7ljWsdNSb5n0TOvZz1HbXtztuhR4jM+Nr87PTYfmx6b7170zOtYSyV5Q5JPJvlEkktX+phOTQoADXhKHAAaEGwAaECwAaABwQaABgQbABoQbABoQLABoIH/AwoY9VG++0QIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_boston\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "df = pd.read_csv('mm_boston_housing.csv') #读取上述处理后的数据\n",
    "y=df['MEDV']\n",
    "X=df.drop(['MEDV','log_MEDV'],axis=1)\n",
    "feat_names = X.columns\n",
    "\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=33,test_size=0.2) #将数据分成训练集和测试集两部分\n",
    "\n",
    "lr = LinearRegression() #最小二乘回归模型\n",
    "lr.fit(X_train,y_train) #训练\n",
    "\n",
    "y_test_pred_lr = lr.predict(X_test) #在测试集上预测\n",
    "y_train_pred_lr = lr.predict(X_train) #在训练集上预测\n",
    "\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef\":list(lr.coef_.T)})\n",
    "fs.sort_values(by=['coef'],ascending=False)\n",
    "\n",
    "print ('The r2 score of LinearRegression on test is', r2_score(y_test,y_test_pred_lr)) #模型训练结果评分\n",
    "print ('The r2 score of LinearRegression on train is', r2_score(y_train,y_train_pred_lr))\n",
    "\n",
    "f,ax = plt.subplots(figsize=(7,5))\n",
    "f.tight_layout()\n",
    "ax.hist(y_train-y_train_pred_lr,bins=40,color='b',alpha=0.5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3)训练岭回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of RidgeCV on test is 0.6961684813070569\n",
      "The r2 score of RidgeCV on train is 0.7548524440445561\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de7xVdZ3/8dcbEMgbJheVi4KBKWreTpiXSmUstBAnsbAZL0WD1dD8muaS/hrLtPmVM85YBOlgWMY0eZuso2JUAilkxEFQPFzkgCCHg3K4iFzk/vn9sdeW7WYfzoJz2fuc/X4+Hudx1vqu71r7sxaH/fns7157fxURmJlZ+elQ7ADMzKw4nADMzMqUE4CZWZlyAjAzK1NOAGZmZapTsQM4GD169Ij+/fsXOwwzszZl7ty56yKiZ357m0oA/fv3p6qqqthhmJm1KZJWFmr3EJCZWZlyAjAzK1NOAGZmZcoJwMysTKVKAJKGSVoiqUbSLQW2d5H0cLJ9tqT+SfvlkuZKWpD8vixnn+uS9pck/UZSj+Y6KTMza1yjCUBSR2ACcAUwGLhO0uC8bqOBjRExELgHuCtpXwcMj4gzgRuByckxOwE/AC6NiA8ALwFjm346ZmaWVppXAEOAmohYHhE7gYeAEXl9RgAPJsuPAUMlKSLmRURd0l4NdJXUBVDyc4QkAUcDdZiZWatJkwD6AKty1muTtoJ9ImI3sAnontfnGmBeROyIiF3Al4AFZJ74BwOTDjp6M7N2rrpuE+OeWcrWHbub/dhpEoAKtOVPInDAPpJOJzMsdHOyfhiZBHAO0JvMENCtBR9cGiOpSlJVfX19inDNzNqP7/9+KT9+bjm79zb/3C1pEkAt0C9nvS/7D9e80ycZ3+8GbEjW+wKPAzdExLKk/9kAEbEsMjPSPAJcWOjBI2JiRFREREXPnvt9ktnMrN16efUmfrfwDUZffDLd3nNYsx8/TQKYAwySNEBSZ2AUUJnXp5LMm7wAI4FpERGSjgGeAm6NiFk5/VcDgyVln9EvBxYd6kmYmbVH455ZylFdO3HTRf1b5PiNfhdQROyWNBaYCnQEHoiIakl3AFURUUlm/H6ypBoylf+oZPexwEDgNkm3JW0fi4g6Sd8GnpW0C1gJ3NScJ2Zm1pZV123itwvf4Kt/MahFqn8AtaU5gSsqKsJfBmdm5eDmyVX8cdl6Zn79siYnAElzI6Iiv92fBDYzKzHVdZuYWv0Goy8e0GLVPzgBmJmVnOzY/+cuGtCij+MEYGZWQhbWvcXU6jf4/EUtW/2DE4CZWUnJVv+fv7hlq39wAjAzKxmL1rzFb6pf53OtUP2DE4CZWckY98xSjurSidEtPPaf5QRgZlYCFq15i6dffp3PXTyAboe3fPUPTgBmZiWhtat/cAIwMyu6xa8n1f9F/Vut+gcnADOzostW/61x508uJwAzsyJa/PpbTFnwOjdd1J9jDu/cqo/tBGBmVkQ/fKaGI7t0YnQrV//gBGBmVjRLXt/MUwvW8LkiVP/gBGBmVjTjnllatOofnADMzIpiyeubmfLyGm66sDjVPzgBmJkVxbhpSzmic/Gqf0iZACQNk7REUo2kWwps7yLp4WT7bEn9k/bLJc2VtCD5fVnOPp0lTZT0iqTFkq5prpMyMytlr7yxmSkL1nDjhSfx3iOKU/1DiikhJXUEJpCZt7cWmCOpMiIW5nQbDWyMiIGSRgF3AZ8B1gHDkykgzyAzrWSfZJ9vAGsj4hRJHYBjm+2szMxK2LhnlnL4YR35wsUnFzWONK8AhgA1EbE8InYCDwEj8vqMAB5Mlh8DhkpSRMyLiLqkvRroKqlLsv554LsAEbE3ItY15UTMzNqCV97I3Plz00X9i1r9Q7oE0AdYlbNey74qfr8+EbEb2AR0z+tzDTAvInZIOiZpu1PSC5IelXRcoQeXNEZSlaSq+vr6FOGamZWuUqn+IV0CUIG2/JnkD9hH0ulkhoVuTpo6AX2BWRFxLvA8cHehB4+IiRFREREVPXv2TBGumVlpWppU/zdeWPzqH9IlgFqgX856X6CuoT6SOgHdgA3Jel/gceCGiFiW9F8PbEvaAR4Fzj2E+M3M2oxx02oy1f+Hi1/9Q7oEMAcYJGmApM7AKKAyr08lcGOyPBKYFhGRDPU8BdwaEbOynSMigCeAS5KmoUDum8pmZu3K0jc28+RLddxwYX+OLYHqH1IkgGRMfyyZO3gWAY9ERLWkOyRdlXSbBHSXVAN8DcjeKjoWGAjcJml+8tMr2fZ14HZJLwHXA//QbGdlZlZixk2r4T2HdeRvSqT6B1CmGG8bKioqoqqqqthhmJkdlJq1m7n8nmf54kffx9eHndrqjy9pbkRU5Lf7k8BmZi1s3DOlV/2DE4CZWYuqWbuZJ16q44YLSmfsP8sJwMysBf3wnbH/4n3nT0OcAMzMWkjN2i1UvljH9RecRPcjuzS+QytzAjAzayE/nLaUrp06MqbExv6znADMzFpAzdotPPFiHTdcWJrVPzgBmJm1iPHTltKlhKt/cAIwM2t2y+ozY/83lOjYf5YTgJlZMxs/rYYunTryNx8p3eofnADMzJrVsvot/Hr+aq6/4CR6lHD1D04AZmbNavy0Gjp36sCYEq/+wQnAzKzZLE+q/xsu6F/y1T84AZiZNZts9V9q3/nTECcAM7NmsLx+C7+av5rrP3QSPY8q/eofnADMzJrF+OnZsf/3FTuU1JwAzMya6NV1W/nVvNX89fltp/qHlAlA0jBJSyTVSLqlwPYukh5Ots+W1D9pv1zSXEkLkt+XFdi3UtLLTT0RM7Ni+eG0pZnq/6NtY+w/q9EEIKkjMAG4AhgMXCdpcF630cDGiBgI3APclbSvA4ZHxJlk5gyenHfsTwFbmnQGZmZFtGLdVn49v46/Pv8keh3VtdjhHJQ0rwCGADURsTwidgIPASPy+owAHkyWHwOGSlJEzIuIuqS9GugqqQuApCPJzB/8naaehJlZsfxwWg2dOqjNVf+QLgH0AVblrNcmbQX7JJPIbwK65/W5BpgXETuS9TuB/wC2HejBJY2RVCWpqr6+PkW4ZmatY8W6rfxq/mr++kNtr/qHdAlABdryZ5I/YB9Jp5MZFro5WT8bGBgRjzf24BExMSIqIqKiZ8+eKcI1M2sd46dnqv+b22D1D+kSQC3QL2e9L1DXUB9JnYBuwIZkvS/wOHBDRCxL+l8AnCdpBTATOEXSjEM7BTOz1rdi3VYen9d2q39IlwDmAIMkDZDUGRgFVOb1qSTzJi/ASGBaRISkY4CngFsjYla2c0TcGxG9I6I/cDHwSkRc0rRTMTNrPW29+ocUCSAZ0x8LTAUWAY9ERLWkOyRdlXSbBHSXVEPmjd3sraJjgYHAbZLmJz+9mv0szMxa0cr1mer/r9rgnT+5FJE/nF+6KioqoqqqqthhmFmZ+6dHX6TyxTqe++dL6XV06ScASXMjoiK/3Z8ENjM7CCvXb+WX81bz2fNPbBNP/gfiBGBmdhAmJGP/X/po2/nOn4Y4AZiZpfTa+m387wvto/oHJwAzs9TGT19Kxw7ii+2g+gcnADOzVF5bv41fvrCazw45kePaQfUPTgBmZqlMmF5Dhw7iS5e0j+ofnADMzBq1asM2/veF2nZV/YMTgJlZo7LVf3sZ+89yAjAzO4BVG7bx2NxM9X98t/ZT/YMTgJnZAU2YXkMHtb/qH5wAzMwalK3+rxvSr91V/+AEYGbWoB/NyFT/X7pkYLFDaRFOAGZmBazasI1Hq2oZ1U6rf3ACMDMr6EczliXVf/sb+89yAjAzy1O7cRuPVq1i1JB+nNDtPcUOp8U4AZiZ5Zkwvf1X/5AyAUgaJmmJpBpJtxTY3kXSw8n22ZL6J+2XS5oraUHy+7Kk/XBJT0laLKla0vea86TMzA5V7cZtPDZ3FZ/5YPuu/iFFApDUEZgAXAEMBq6TNDiv22hgY0QMBO4B7kra1wHDI+JMMnMGT87Z5+6IOBU4B7hI0hVNOhMzs2bwoxnLEO2/+od0rwCGADURsTwidgIPASPy+owAHkyWHwOGSlJEzIuIuqS9GugqqUtEbIuI6QDJMV8A+jb1ZMzMmmL1m2/zaNUqPv3BvvQ+pn1X/5AuAfQBVuWs1yZtBfskk8hvArrn9bkGmBcRO3IbJR0DDAeeKfTgksZIqpJUVV9fnyJcM7ND86PpNQB8uZ3e958vTQJQgbb8meQP2EfS6WSGhW5+105SJ+AXwLiIWF7owSNiYkRURERFz549U4RrZnbwVr/5No9UZcb+y6H6h3QJoBbol7PeF6hrqE/ypN4N2JCs9wUeB26IiGV5+00ElkbE9w8+dDOz5pOt/tvrp34LSZMA5gCDJA2Q1BkYBVTm9akk8yYvwEhgWkREMrzzFHBrRMzK3UHSd8gkiq825QTMzJqqLqn+P13Rjz5lUv1DigSQjOmPBaYCi4BHIqJa0h2Srkq6TQK6S6oBvgZkbxUdCwwEbpM0P/nplbwq+AaZu4peSNq/0LynZmaWzo9mJGP/l5ZP9Q/QKU2niJgCTMlr+2bO8nbg2gL7fQf4TgOHLfS+gZlZq6p7820enrOKa8us+gd/EtjMyty9MzJvTX65DO77z+cEYGZlK7f67/vew4sdTqtzAjCzsnXvjGUEUZbVPzgBmFmZWrMpU/2PPK88q39wAjCzMnXvjGXsjeBvLy3P6h+cAMysDK3Z9DYP/XkV11b0LdvqH5wAzKwM3ZdU/+XynT8NcQIws7Ly+qbt/CKp/vsdW77VPzgBmFmZuXdGjav/hBOAmZWN1zdt5xdzVjHyPFf/4ARgZmXkvj8sY+/e4G/L7Dt/GuIEYGZl4Y23tvM/f36Na8519Z/lBGBmZeHeGa7+8zkBmFm7l1v9n9jd1X+WE4CZtXv3zljGHlf/+0mVACQNk7REUo2kWwps7yLp4WT7bEn9k/bLJc2VtCD5fVnOPucl7TWSxkny/ABm1uzWvrWdX/z5Na45t4+r/zyNJgBJHYEJwBVkZvC6TtLgvG6jgY0RMRC4h8wE8ADrgOERcSaZKSMn5+xzLzAGGJT8DGvCeZiZFXTvH5axe28w9tJBxQ6l5KR5BTAEqImI5RGxE3gIGJHXZwTwYLL8GDBUkiJiXkRkJ5CvBromrxZOAI6OiOcjIoCfAVc3+WzMzHKsfWs7/zP7NT51jqv/QtIkgD7Aqpz12qStYJ9kDuFNQPe8PtcA8yJiR9K/tpFjAiBpjKQqSVX19fUpwjUzy7jvD8sz1f9lHvsvJE0CKDQ2HwfTR9LpZIaFbj6IY2YaIyZGREVEVPTs2TNFuGZmmer/57NX8qlz+nBS9yOKHU5JSpMAaoF+Oet9gbqG+kjqBHQDNiTrfYHHgRsiYllO/76NHNPM7JC5+m9cmgQwBxgkaYCkzsAooDKvTyWZN3kBRgLTIiIkHQM8BdwaEbOynSNiDbBZ0oeSu39uAH7dxHMxMwNg7eZM9f+Xrv4PqNEEkIzpjwWmAouARyKiWtIdkq5Kuk0CukuqAb4GZG8VHQsMBG6TND/56ZVs+xLwY6AGWAY83VwnZWbl7b+y1b/v+z8gZW7CaRsqKiqiqqqq2GGYWQlbu3k7H75rOsPP6s3d155V7HBKgqS5EVGR3+5PAptZuzLR1X9qTgBm1m6s3byd/569kqvP7kP/Hh77b4wTgJm1GxP/sJxde4Kv+M6fVJwAzKxdqN+8g/+evZIRZ/d29Z+SE4CZtQsTn13Gzt17+cpl/s6ftJwAzKzNq9+8g8l/WsnV5/RhgKv/1JwAzKzNu/+55a7+D4ETgJm1aeu27OBnz6/g6rNd/R8sJwAza9MmPpup/v2dPwfPCcDM2qx1W3Yw+fmVjDi7Dyf3PLLY4bQ5TgBm1mbd/+xyduze4+r/EDkBmFmblBn7z1T/73P1f0icAMysTbr/OVf/TeUEYGZtzvotO/jZH1dy1Vm9Xf03gROAmbU5E9+p/n3ff1M4AZhZm7I+ufNn+Fm9GdjL1X9TpEoAkoZJWiKpRtItBbZ3kfRwsn22pP5Je3dJ0yVtkTQ+b5/rJC2Q9JKk30jq0RwnZGbt2/3Pvcrbu/b4U7/NoNEEIKkjMAG4AhgMXCdpcF630cDGiBgI3APclbRvB24D/jHvmJ2AHwCXRsQHgJfITB9pZtagDVt38rPnV3CVq/9mkeYVwBCgJiKWR8RO4CFgRF6fEcCDyfJjwFBJioitETGTTCLIpeTniGRS+KOBukM9CTMrD/c/tzyp/n3nT3NIkwD6AKty1muTtoJ9kknkNwHdGzpgROwiMyn8AjJP/IPJTCy/H0ljJFVJqqqvr08Rrpm1Rxu27uTBP65g+Ad6M7DXUcUOp11IkwBUoC1/Jvk0ffZ1lg4jkwDOAXqTGQK6tVDfiJgYERURUdGzZ88U4ZpZe5St/v9uqKv/5pImAdQC/XLW+7L/cM07fZLx/W7AhgMc82yAiFgWEQE8AlyYMmYzKzPZ6v+Trv6bVZoEMAcYJGmApM7AKKAyr08lcGOyPBKYljyxN2Q1MFhStqS/HFiUPmwzKyc/zlb/HvtvVp0a6xARuyWNBaYCHYEHIqJa0h1AVURUkhm/nyyphkzlPyq7v6QVZN7k7SzpauBjEbFQ0reBZyXtAlYCNzXvqZlZe5Bb/Q86ztV/c2o0AQBExBRgSl7bN3OWtwPXNrBv/wba7wPuSxuomZWnHz+3nG2u/luEPwlsZiVrY1L9f+LME1z9twAnADMrWT+emVT/Q/2p35bgBGBmJWnj1p38dNYKrjzzBE5x9d8inADMrCRNmvlqMvbv6r+lOAGYWcnZuHUnP/1jpvp///Gu/luKE4CZlZxJM19ly47drv5bmBOAmZWUN7dlqv9PuPpvcU4AZlZS3qn+fedPi3MCMLOS8ea2nfxk1gquPPN4V/+twAnAzErGA67+W5UTgJmVhNzq/9Tjjy52OGXBCcDMSsIDM19ls6v/VuUEYGZFt2nbLn4yawVXnOHqvzU5AZhZ0U2a5eq/GJwAzKyoNm3bxU9mvsqw04/ntBNc/bcmJwAzK6oHXP0XTaoEIGmYpCWSaiTdUmB7F0kPJ9tnS+qftHeXNF3SFknj8/bpLGmipFckLZZ0TXOckJm1HZve3sUDszLV/+Derv5bW6MzgknqCEwgM29vLTBHUmVELMzpNhrYGBEDJY0C7gI+A2wHbgPOSH5yfQNYGxGnSOoAHNvkszGzNuWBma+yebur/2JJ8wpgCFATEcsjYifwEDAir88I4MFk+TFgqCRFxNaImEkmEeT7PPBdgIjYGxHrDukMzKxNylb/Hz/9OFf/RZImAfQBVuWs1yZtBftExG5gE9C9oQNKOiZZvFPSC5IelXRcA33HSKqSVFVfX58iXDNrC34yy9V/saVJACrQFofQJ1cnoC8wKyLOBZ4H7i7UMSImRkRFRFT07NkzRbhmVuo2vb2LSTMz1f/pvbsVO5yylSYB1AL9ctb7AnUN9ZHUCegGbDjAMdcD24DHk/VHgXNTxGJm7cBPZ61w9V8C0iSAOcAgSQMkdQZGAZV5fSqBG5PlkcC0iGjwFUCy7QngkqRpKLCwof5m1n5kqv/lfGywq/9ia/QuoIjYLWksMBXoCDwQEdWS7gCqIqISmARMllRDpvIfld1f0grgaKCzpKuBjyV3EH092ef7QD3wueY9NTMrNWs2vc1tv3qZt1z9l4RGEwBAREwBpuS1fTNneTtwbQP79m+gfSXwkbSBmlnbtX3XHibNfJXx02rYG8H/vfJUzujj6r/YUiUAM7NDERH8ftFa7nxyIa9t2MYVZxzP/73yNPode3ixQzOcAMyshdSs3cIdTy7k2VfqGdTrSH7+hfO5aGCPYodlOZwAzKxZbd6+i3HPLOUns1bwns4d+eYnB3P9BSdxWEd/9VipcQIws2axd2/wy3mr+d7Ti1m/dQefqejHP378/fQ4skuxQ7MGOAGYWZO9uOpNvlVZzfxVb3LOicfwwE0VfKDvMY3vaEXlBGBmh6x+8w7+fepiHqmqpedRXfjPT5/F1Wf3oUOHQl8OYKXGCcDMDtquPXt58I8r+MHvl7J99x5u/sjJfGXoII7s4qeUtsT/WmZ2UJ5bWs+3n1hIzdotfPSUnnxz+GDe1/PIYodlh8AJwMxSWbVhG995aiFTq9/gpO6HM+nGCi47tReSh3vaKicAMzugt3fu4d4ZNdz37HI6dRD/POz9jL54AF06dSx2aNZETgBmVlBE8NSCNfy/pxZRt2k7I87uza1XnMbx3boWOzRrJk4AZrafRWve4vbKama/uoHBJxzND647hw/296yt7Y0TgJm9481tO/nP373Cf/9pJd3ecxj/+pdnMOqDJ9LRt3W2S04AZsaevcFDc17j7qlL2PT2Lq7/0En8/eWncMzhnYsdmrUgJwCzMjdnxQa+9etqFq55i/MHHMvtV53OaSd4kvZy4ARgVqZe37Sd7z69iF/Pr6N3t66M/+w5fOLME3xbZxlJ9fV8koZJWiKpRtItBbZ3kfRwsn22pP5Je3dJ0yVtkTS+gWNXSnq5KSdhZunt2L2HCdNruOw/ZvD0y6/zd5cN5Pf/8FE++YHefvIvM42+ApDUEZgAXE5m8vc5kiqTaR2zRgMbI2KgpFHAXcBngO3AbcAZyU/+sT8FbGnyWZhZoyKCaYvXcseTC1m5fhsfP/04/uUTgz05SxlLMwQ0BKiJiOUAkh4CRvDuSdxHALcny48B4yUpIrYCMyUNzD+opCOBrwFjgEcO+QzMrFHL6rdw55MLmbGknoG9jmTy6CF8eFDPYodlRZYmAfQBVuWs1wLnN9QnmUR+E9AdWHeA494J/Aew7UAPLmkMmSTBiSeemCJcM8vavH0X46fV8MCsV+naqSP/8onTuPHC/p6cxYB0CaDQoGAcQp99naWzgYER8ffZ9wsaEhETgYkAFRUVDR7TzPbZuzd4fN5qvvebxdRv3sGnK/ryTx8/lZ5HeXIW2ydNAqgF+uWs9wXqGuhTK6kT0A3YcIBjXgCcJ2lFEkMvSTMi4pKUcZtZA16qfZPbK6t54bU3ObvfMdx/QwVn9/PkLLa/NAlgDjBI0gBgNTAK+Gxen0rgRuB5YCQwLSIarNYj4l7gXoDkFcCTfvI3a5p1W3Zw99QlPFy1iu5HdOHua8/iU+d4chZrWKMJIBnTHwtMBToCD0REtaQ7gKqIqAQmAZMl1ZCp/Edl90+q/KOBzpKuBj6WdweRmTXBrj17mfz8Su75/Su8vXMPX7h4AF8ZOoijux5W7NCsxOkAhXrJqaioiKqqqmKHYVYyZtWs4/bKapau3cKHB/XgW8NPZ2AvT85i7yZpbkRU5Lf7k8BmbdCqDdv416cW8Zvq1znx2MO5/4YK/uI0T85iB8cJwKwNeXvnHu77wzLu+8MyOkj808czk7N0PcyTs9jBcwIwawMigqdffp1/fWoRq998m6vO6s2tV57KCd3eU+zQrA1zAjArcUte38ztldU8v3w9px5/FA+P+RDnn9y92GFZO+AEYFaiNm3bxT2/f4XJf1rJUV07cefVZ3DdB/vRyZ/itWbiBGBWYvbsDR6pWsW/T13Cm9t28lfnn8TXLj+F9x7hyVmseTkBmJWQuSs38K3Kal5e/RZDBhzL7cNPZ3BvT85iLcMJwKwEvPHWdr739GIen7ea44/uyrjrzmH4Bzw5i7UsJwCzItqxew8PzFzBD6ctZfeeYOylA/nype/j8M7+r2ktz39lZkUybfEb3PHEQlas38blg4/jXz5xGid1P6LYYVkZcQIwa2WvrtvKnU8uZNritZzc8wge/PwQPnqKJ2ex1ucEYNZKtuzYzfhpNUyauZwunTryjSszk7N07uTbOq04nADMWlhE8Kv5q/nulMWs3byDkef15Z+HvZ9eR3UtdmhW5pwAzFrQy6s38a3Kauau3MhZfbvxX9efxzknvrfYYZkBTgBmLWL9lh3c/dtXeGjOa3Q/ojP/NvIDjDy3rydnsZKSavBR0jBJSyTVSLqlwPYukh5Ots/OzvMrqbuk6ZK2SBqf0/9wSU9JWiypWtL3muuEzIpp9569/HTWq1x69wwerVrF5y8awLR/vIRPV/Tzk7+VnEZfAUjqCEwALicz9+8cSZV5s3qNBjZGxEBJo4C7gM8A24HbgDOSn1x3R8R0SZ2BZyRdERFPN/2UzIrjjzXr+PYTC1nyxmYuHtiDbw0fzKDjjip2WGYNSjMENASoiYjlAJIeAkYAuQlgBHB7svwYMF6SImIrMFPSwNwDRsQ2YHqyvFPSC2Qmm28RW3fsZm8EksjWYBIIkftBy2zbvmXe+SSmkjZy2qzpIoII2BtBkPwOGmgL9ub+JmffSLkvwd69mXZ49757AyD7GNm2gGDfvkk7OY+3J4Jfz1/NlAWv0/e97+G/rj+Pjw0+zn8nVvLSJIA+wKqc9Vrg/Ib6JHMIbwK6A+saO7ikY4DhwA8a2D4GGANw4oknpgh3f1dPmMXStVsOad8DeSchQMHkwru2Hzi55HTPHOsAx4b87Q0f+522d/rrnT7kHjvv8WD/J8/c9eyTa/bJ8l1tezPL++3Lu5/w97ad2Ugb1fWwDvzD5afwNx852ZOzWJuRJgEUKmPy/+um6bP/gaVOwC+AcdlXGPsdJGIiMBEycwI3dsxCbv7o+9i4dSeRhJQUcGSnQ85WkjmP+U6fff3fvW92h2hge+6xedexCh87//HTHBv2Vby557Ff3O9s3/dE/e7j7Xs8co7VQUISHZKEkV2XoIOy6/uSSIds32w7mfUOHbQvkeXu+662Bvbd7zEKHS8b476+B44vr61DA/uyb115+3bI2V+CE7p1pfuRXQr+/ZmVqjQJoBbol7PeF6hroE9t8qTeDdiQ4tgTgaUR8f0UfQ/ZyPNabHTJzKzNSnMX0BxgkKQByRu2o4DKvD6VwI3J8khgWkRuTbs/Sd8hkyi+enAhm5lZc2j0FUAypj8WmAp0BB6IiGpJdwBVEVEJTAImS6ohU/mPyu4vaQVwNNBZ0tXAx4C3gG8Ai4EXkjHn8RHx4+Y8OTMza1iqD4JFxBRgSl7bN3OWtwPXNrBv/wYO61skzMyKyN9CZWZWppwAzMzKlJLIHhQAAAUGSURBVBOAmVmZcgIwMytTTgBmZmVKjdyuX1Ik1QMrD3H3HqT4aooicFwHx3EdHMd1cNprXCdFxH7zjrapBNAUkqoioqLYceRzXAfHcR0cx3Vwyi0uDwGZmZUpJwAzszJVTglgYrEDaIDjOjiO6+A4roNTVnGVzXsAZmb2buX0CsDMzHI4AZiZlal2mwAk/bukxZJekvR4MvVkoX7DJC2RVCPpllaI61pJ1ZL2Smrwti5JKyQtkDRfUlUJxdXa1+tYSb+TtDT5/d4G+u1JrtV8SfnzVTRnPAc8f0ldJD2cbJ8tqX9LxXKQcd0kqT7nGn2hFWJ6QNJaSS83sF2SxiUxvyTp3JaOKWVcl0jalHOtvlmoXwvE1U/SdEmLkv+L/6dAn+a9ZpkpCtvfD5l5Bzoly3cBdxXo0xFYBpwMdAZeBAa3cFynAe8HZgAVB+i3AujRiter0biKdL3+DbglWb6l0L9jsm1LK1yjRs8f+DJwX7I8Cni4ROK6icycG63y95Q85keAc4GXG9h+JfA0ma+G/xAwu0TiugR4sjWvVfK4JwDnJstHAa8U+Hds1mvWbl8BRMRvI2J3svonMlNZ5hsC1ETE8ojYCTwEjGjhuBZFxJKWfIxDkTKuVr9eyfEfTJYfBK5u4cc7kDTnnxvvY8BQJTMeFTmuVhcRz3LgqWFHAD+LjD8Bx0g6oQTiKoqIWBMRLyTLm4FFQJ+8bs16zdptAsjzeTJZM18fYFXOei37X/BiCeC3kuZKGlPsYBLFuF7HRcQayPwHAXo10K+rpCpJf0pmnmsJac7/nT5JAbIJ6N5C8RxMXADXJMMGj0nqV2B7ayvl/38XSHpR0tOSTm/tB0+GDs8BZudtatZrlmpGsFIl6ffA8QU2fSMifp30+QawG/h5oUMUaGvyfbFp4krhooiok9QL+J2kxUnlUsy4Wv16HcRhTkyu18nANEkLImJZU2PLk+b8W+QaNSLNYz4B/CIidkj6IplXKZe1cFyNKca1SuMFMt+ds0XSlcCvgEGt9eCSjgT+F/hqRLyVv7nALod8zdp0AoiIvzjQdkk3Ap8EhkYygJanFsithPoCdS0dV8pj1CW/10p6nMzL/CYlgGaIq9Wvl6Q3JJ0QEWuSl7prGzhG9notlzSDTPXU3Akgzfln+9RK6gR0o+WHGxqNKyLW56zeT+Z9sWJrkb+npsp90o2IKZJ+JKlHRLT4l8RJOozMk//PI+KXBbo06zVrt0NAkoYBXweuiohtDXSbAwySNEBSZzJv2rXYHSRpSTpC0lHZZTJvaBe8Y6GVFeN6VQI3Jss3Avu9UpH0XkldkuUewEXAwhaIJc3558Y7EpjWQPHRqnHljRNfRWZ8udgqgRuSO1s+BGzKDvcVk6Tjs+/bSBpC5nly/YH3apbHFTAJWBQR/9lAt+a9Zq39Tndr/QA1ZMbK5ic/2TszegNTcvpdSebd9mVkhkJaOq6/JJPFdwBvAFPz4yJzN8eLyU91qcRVpOvVHXgGWJr8PjZprwB+nCxfCCxIrtcCYHQLxrPf+QN3kCk0ALoCjyZ/f38GTm7pa5Qyru8mf0svAtOBU1shpl8Aa4Bdyd/WaOCLwBeT7QImJDEv4AB3xbVyXGNzrtWfgAtbKa6LyQznvJTzvHVlS14zfxWEmVmZardDQGZmdmBOAGZmZcoJwMysTDkBmJmVKScAM7My5QRgZlamnADMzMrU/wf4fWfryZxY7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the best alpha is  0.1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "      <th>coef_ridge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>0.452377</td>\n",
       "      <td>0.449104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>0.129239</td>\n",
       "      <td>0.125966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>RAD_24</td>\n",
       "      <td>0.103221</td>\n",
       "      <td>0.099911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>B</td>\n",
       "      <td>0.078899</td>\n",
       "      <td>0.079327</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>0.059660</td>\n",
       "      <td>0.060075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>RAD_7</td>\n",
       "      <td>0.037664</td>\n",
       "      <td>0.036712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>RAD_8</td>\n",
       "      <td>0.030071</td>\n",
       "      <td>0.030423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>RAD_3</td>\n",
       "      <td>0.027465</td>\n",
       "      <td>0.028356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>0.013818</td>\n",
       "      <td>0.011491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>-0.001228</td>\n",
       "      <td>-0.001077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>RAD_4</td>\n",
       "      <td>-0.008698</td>\n",
       "      <td>-0.008646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>RAD_5</td>\n",
       "      <td>-0.011517</td>\n",
       "      <td>-0.011183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>RAD_2</td>\n",
       "      <td>-0.040970</td>\n",
       "      <td>-0.039042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>RAD_6</td>\n",
       "      <td>-0.055736</td>\n",
       "      <td>-0.055449</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>RAD_1</td>\n",
       "      <td>-0.081501</td>\n",
       "      <td>-0.081083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>TAX</td>\n",
       "      <td>-0.113548</td>\n",
       "      <td>-0.110192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>-0.151553</td>\n",
       "      <td>-0.145910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>PTRATIO</td>\n",
       "      <td>-0.187898</td>\n",
       "      <td>-0.187369</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>-0.221874</td>\n",
       "      <td>-0.212619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>-0.386249</td>\n",
       "      <td>-0.374867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>-0.476675</td>\n",
       "      <td>-0.474603</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    columns      coef  coef_ridge\n",
       "5        RM  0.452377    0.449104\n",
       "1        ZN  0.129239    0.125966\n",
       "20   RAD_24  0.103221    0.099911\n",
       "10        B  0.078899    0.079327\n",
       "3      CHAS  0.059660    0.060075\n",
       "18    RAD_7  0.037664    0.036712\n",
       "19    RAD_8  0.030071    0.030423\n",
       "14    RAD_3  0.027465    0.028356\n",
       "2     INDUS  0.013818    0.011491\n",
       "6       AGE -0.001228   -0.001077\n",
       "15    RAD_4 -0.008698   -0.008646\n",
       "16    RAD_5 -0.011517   -0.011183\n",
       "13    RAD_2 -0.040970   -0.039042\n",
       "17    RAD_6 -0.055736   -0.055449\n",
       "12    RAD_1 -0.081501   -0.081083\n",
       "8       TAX -0.113548   -0.110192\n",
       "4       NOX -0.151553   -0.145910\n",
       "9   PTRATIO -0.187898   -0.187369\n",
       "0      CRIM -0.221874   -0.212619\n",
       "7       DIS -0.386249   -0.374867\n",
       "11    LSTAT -0.476675   -0.474603"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_boston\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import RidgeCV\n",
    "\n",
    "df = pd.read_csv('mm_boston_housing.csv')#读取上述处理后的数据\n",
    "\n",
    "y=df['MEDV']\n",
    "X=df.drop(['MEDV','log_MEDV'],axis=1)\n",
    "feat_names = X.columns\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=33,test_size=0.2)#将数据分成训练集和测试集两部分\n",
    "\n",
    "alphas = [0.01,0.1,1,10,100]\n",
    "ridge = RidgeCV(alphas = alphas, store_cv_values=True)#岭回归模型\n",
    "ridge.fit(X_train,y_train)\n",
    "\n",
    "y_test_pred_ridge = ridge.predict(X_test)#在测试集上预测\n",
    "y_train_pred_ridge = ridge.predict(X_train)#在训练集上预测\n",
    "\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef\":list(ridge.coef_.T)})\n",
    "fs.sort_values(by=['coef'],ascending=False)\n",
    "\n",
    "print ('The r2 score of RidgeCV on test is', r2_score(y_test,y_test_pred_ridge))#模型训练结果评分\n",
    "print ('The r2 score of RidgeCV on train is', r2_score(y_train,y_train_pred_ridge))\n",
    "\n",
    "\n",
    "mse_mean = np.mean(ridge.cv_values_,axis=0)\n",
    "plt.plot(np.log10(alphas),mse_mean.reshape(len(alphas),1))\n",
    "plt.show()\n",
    "\n",
    "print('the best alpha is ',ridge.alpha_)\n",
    "\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef\":list(lr.coef_.T),\"coef_ridge\":list(ridge.coef_.T)})\n",
    "fs.sort_values(by=['coef'],ascending=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4)训练lasso回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of lasso on test is 0.6953719445869475\n",
      "The r2 score of lasso on train is 0.7546505272258799\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de3gc9X3v8fdXd8vWxbbkq4xlx8YXLiYgDAkmFwiJkxJME5I4oYEmtKRpSduT5uSQ5IQmac5pSdtD2yec9BCgJSQppBRSt4EYOIQEUrAtYxvji4x8l2TLkiXrYlnX/faPHcGylayVLO3saj+v59lHszO/nf3+GDOf3d/Mzpi7IyIimScr7AJERCQcCgARkQylABARyVAKABGRDKUAEBHJUDlhFzAaZWVlXllZGXYZIiJpZevWrc3uXh4/P60CoLKykurq6rDLEBFJK2Z2eKj5GgISEclQCgARkQylABARyVAKABGRDKUAEBHJUAoAEZEMpQAQEclQCgARkRS2v6mTv366hr6ByLivWwEgIpKievoH+MKPt/HDlw/Terp33NefVr8EFhHJJHc/VcPuY+3cf0sVs4oLxn39+gYgIpKCfrH3BA/++iC//c5K3rdy9oS8hwJARCTFnGjv5kv/vIPlc4q484PLJ+x9NAQkIpJi7vvVATq6+3n0c1dSkJs9Ye+jbwAiIinmlSOtrFpQwpJZRRP6PgoAEZEU0jcQYVdDOxdXlE74eykARERSyL7GDnr6I1xcUTLh76UAEBFJITuOtgGwSt8AREQyy6t1pyiZksvCmYUT/l4KABGRFLKjro2LK0owswl/LwWAiEiKONM7wL7GjqQM/4ACQEQkZew+1sZAxJNyABgUACIiKeONA8AL9A1ARCSjvFp3itnF+cyegAu/DUUBICKSIqIHgJPz6R8SDAAzW2tmNWZWa2Z3DrE838weDZZvMrPKuOXnmVmnmX0p0XWKiGSStjN9HGw+zaokjf9DAgFgZtnAvcAHgZXAJ81sZVyz24BWd18C3APcHbf8HuCpUa5TRCRj7KxL7vg/JPYNYDVQ6+4H3L0XeARYF9dmHfBQMP0YcK0FJ7Ga2Y3AAWDXKNcpIpIxdtSdAuDi+akVAPOBozHP64J5Q7Zx936gDZhpZlOB/wF8cwzrBMDMbjezajOrbmpqSqBcEZH0s6uhjfNmFFJSmJu090wkAIb6OZon2OabwD3u3jmGdUZnut/n7lXuXlVeXj5isSIi6WjPsQ5Wzi1O6nsmckOYOmBBzPMKoGGYNnVmlgOUAC3AFcBNZvYdoBSImFk3sDWBdYqIZISu3n4OnTzNjZcMORAyYRIJgC3AUjNbBNQD64FPxbXZANwKvATcBDzn7g5cPdjAzL4BdLr7d4OQGGmdIiIZoeZ4B+6wfO7E3gAm3ogB4O79ZnYHsBHIBh50911m9i2g2t03AA8AD5tZLdFP/uvHss5z7IuISFrac6wDICWHgHD3J4En4+bdFTPdDXxshHV8Y6R1iohkor3H25mWn0PF9ClJfV/9ElhEJGR7jrWzfE5RUi4BHUsBICISIndn77EOViR5+AcUACIioaprPUNHT3/SDwCDAkBEJFR7j0cPAOsbgIhIhtlzrB0zWDZb3wBERDLK3uPtLJxRyNT8hE7KHFcKABGREO0J6QAwKABEREIzeAmI5XMUACIiGWXwEhArQjgDCBQAIiKhGbwEhIaAREQyzN7j7RSFcAmIQQoAEZGQvFbfxoq5xUm/BMQgBYCISAj6BiLsamhn1YLk3QQ+ngJARCQENcc76OmPcHFF8u4BHE8BICISglfr2gC4ZIECQEQko+w4eorphbmhHQAGBYCISCh21J1i1YLS0A4AgwJARCTpunr72dfYEer4PygARESS7rX6diIOl4R4BhAoAEREku7VulMA+gYgIpJpth89xfzSKZRNyw+1DgWAiEiSvVrXFuoPwAYpAEREkqjldC9HWrpYFfLwDyQYAGa21sxqzKzWzO4cYnm+mT0aLN9kZpXB/NVmtj147DCz34x5zSEz2xksqx6vDomIpLJUGf8HGPEeZGaWDdwLXAfUAVvMbIO7745pdhvQ6u5LzGw9cDfwCeA1oMrd+81sLrDDzP7N3fuD173X3ZvHs0MiIqlsx9E2zOCiivQYAloN1Lr7AXfvBR4B1sW1WQc8FEw/BlxrZubuXTE7+wLAx6NoEZF0taPuFEvKpzEthHsAx0skAOYDR2Oe1wXzhmwT7PDbgJkAZnaFme0CdgK/FxMIDjxtZlvN7Pbh3tzMbjezajOrbmpqSqRPIiIpKRJxth5u5dLzpoddCpBYAAz1O+X4T/LDtnH3Te5+AXA58BUzKwiWX+XulwIfBP7AzN411Ju7+33uXuXuVeXl5QmUKyKSmvad6KDtTB+rF80IuxQgsQCoAxbEPK8AGoZrY2Y5QAnQEtvA3fcAp4ELg+cNwd8TwBNEh5pERCatzQeju8V0CoAtwFIzW2RmecB6YENcmw3ArcH0TcBz7u7Ba3IAzGwhsAw4ZGZTzawomD8VeD/RA8YiIpPW5oMtzC0pCPUKoLFGPAoRnMFzB7ARyAYedPddZvYtoNrdNwAPAA+bWS3RT/7rg5evAe40sz4gAvy+uzeb2WLgieAqeDnAj9395+PdORGRVOHubD7YwjveNjPUK4DGSugwtLs/CTwZN++umOlu4GNDvO5h4OEh5h8AVo22WBGRdHWkpYsTHT1cXpkawz+gXwKLiCTFpmD8/4oUGf8HBYCISFJsPtjCjKl5LJk1LexS3qAAEBFJgi2HWqhaOD1lxv9BASAiMuGOt3Vz+GRXypz+OUgBICIywTYfSq3z/wcpAEREJtiWgy1Mzctm5dzisEt5CwWAiMgE23TwJJcunE5OdmrtclOrGhGRSaaxvZt9jZ1ctaQs7FL+CwWAiMgEeuH16C1Prl6qABARySgvvt5E2bQ8VsxJrfF/UACIiEyYSMR5sbaZNUvKyMpKnfP/BykAREQmyJ7j7TR39nL10tS8l4kCQERkgqTy+D8oAEREJsyLrzezfE4Rs4oLRm4cAgWAiMgEONM7wOZDLSn76R8UACIiE2LzoRZ6+yOsSdHxf1AAiIhMiBf2NZGXk8XqFLoBTDwFgIjIBHjh9WZWV85gSl522KUMSwEgIjLOjrZ0UdPYwXuWpe7wDygARETG3TO7GwG4buXskCs5OwWAiMg4e3r3cZbNLmLhzKlhl3JWCgARkXHUerqXLYdaU/7TPygARETG1XN7TzAQcd5/wSQJADNba2Y1ZlZrZncOsTzfzB4Nlm8ys8pg/moz2x48dpjZbya6ThGRdPTM7kbmFBdw0fySsEsZ0YgBYGbZwL3AB4GVwCfNbGVcs9uAVndfAtwD3B3Mfw2ocvdLgLXA/zOznATXKSKSVrr7BvjlviauWzkbs9S7+me8RL4BrAZq3f2Au/cCjwDr4tqsAx4Kph8DrjUzc/cud+8P5hcAPop1ioiklV/XNnOmbyAtxv8hsQCYDxyNeV4XzBuyTbDDbwNmApjZFWa2C9gJ/F6wPJF1ioiklad3NVKUn8OVi2eGXUpCEgmAob7HeKJt3H2Tu18AXA58xcwKElxndMVmt5tZtZlVNzU1JVCuiEjyDUSc/7+3kfcsn0VeTnqcX5NIlXXAgpjnFUDDcG3MLAcoAVpiG7j7HuA0cGGC6xx83X3uXuXuVeXlqf2rOhHJXC/tP0lzZy8funBO2KUkLJEA2AIsNbNFZpYHrAc2xLXZANwaTN8EPOfuHrwmB8DMFgLLgEMJrlNEJG38dHs9Rfk5vHf5rLBLSVjOSA3cvd/M7gA2AtnAg+6+y8y+BVS7+wbgAeBhM6sl+sl/ffDyNcCdZtYHRIDfd/dmgKHWOc59ExFJiu6+AX7+2nE+eOEcCnJT9+Jv8UYMAAB3fxJ4Mm7eXTHT3cDHhnjdw8DDia5TRCQdPbf3BJ09/dz49vQ6lyU9jlSIiKSwn26rZ1ZRftqc/TNIASAicg7auvp4vqaJD6+aR3ZW6v/4K5YCQETkHDz12jF6ByLceEl6Df+AAkBE5Jz8dHs9i8umcuH84rBLGTUFgIjIGDWcOsOmgy3ccMm8tLj2TzwFgIjIGP10ez3u8JG3V4RdypgoAERExsDdefyVei6vnM55MwvDLmdMFAAiImOws76N2hOdfOTS9Pz0DwoAEZExefyVevJysvjQRXPDLmXMFAAiIqPUNxBhw44Grls5m5IpuWGXM2YKABGRUfplTRMtp3v5SJpd+iGeAkBEZJQe31bHzKl5vOv89L5EvQJARGQUTnX18uzuE9xwyTxys9N7F5re1YuIJNljW+voHYjwscsWjNw4xSkAREQSFIk4P3z5MFULp7NyXvpd+iGeAkBEJEEv1DZz6GQXn37HwrBLGRcKABGRBD380mHKpuWxNo3u+3s2CgARkQTUtXbx3N5GPnH5AvJz0ue2j2ejABARScA/bT4CwCdXnxdyJeNHASAiMoKe/gEe2XyUa5bPpmJ6el74bSgKABGREfz7jmOcPN3LLZPk4O8gBYCIyFm4O99/4QDnz57G1UvLwi5nXCkARETO4te1J9l7vIPfWbM4Le/6dTYKABGRs7j/xQOUTcvjhkvmhV3KuEsoAMxsrZnVmFmtmd05xPJ8M3s0WL7JzCqD+deZ2VYz2xn8vSbmNc8H69wePGaNV6dERMbD640dPF/TxC3vqKQgd3Kc+hkrZ6QGZpYN3AtcB9QBW8xsg7vvjml2G9Dq7kvMbD1wN/AJoBn4sLs3mNmFwEYg9vqpN7t79Tj1RURkXN3/wkEKcrP4rSsn18HfQYl8A1gN1Lr7AXfvBR4B1sW1WQc8FEw/BlxrZubu29y9IZi/Cygws/zxKFxEZCI1dfTwxPZ6PnppBTOm5oVdzoRIJADmA0djntfx1k/xb2nj7v1AGzAzrs1HgW3u3hMz7x+C4Z+v2zBHV8zsdjOrNrPqpqamBMoVETl333/hAP0DEW5bsyjsUiZMIgEw1I7ZR9PGzC4gOiz0uZjlN7v7RcDVwePTQ725u9/n7lXuXlVent43XxCR9HCio5sfvHSIGy+Zz+LyaWGXM2ESCYA6IPbC1xVAw3BtzCwHKAFagucVwBPALe6+f/AF7l4f/O0Afkx0qElEJHTfe34/fQPOH167NOxSJlQiAbAFWGpmi8wsD1gPbIhrswG4NZi+CXjO3d3MSoGfAV9x918PNjazHDMrC6ZzgeuB186tKyIi5+54Wzc/2nSEj146n8qyqWGXM6FGDIBgTP8Oomfw7AF+4u67zOxbZnZD0OwBYKaZ1QJfBAZPFb0DWAJ8Pe50z3xgo5m9CmwH6oHvj2fHRETG4t5f1BKJOF+4ZnJ/+ocETgMFcPcngSfj5t0VM90NfGyI130b+PYwq70s8TJFRCZe/akzPLLlCB+/fAELZkyei74NR78EFhEJ/PXGGsyMP3jvkrBLSQoFgIgIsOPoKR7fVs/vrFnE/NIpYZeTFAoAEcl47s63f7absml5fP49bwu7nKRRAIhIxvv5a8fZcqiVL163jKKC3LDLSRoFgIhktJ7+Af78qb0sm13Ex6sqwi4nqRQAIpLRHn7pMEdauvif168gJzuzdomZ1VsRkRjt3X3c+4tarl5axtVLM+9SMwoAEclY9//qAK1dfXz5A8vDLiUUCgARyUjNnT3c/+JBfuPiuVxUURJ2OaFQAIhIRvruc7X09Ef4k+vOD7uU0CgARCTjHG3p4kebDvPxqopJfbnnkSgARCTj3PPsPrLMJv3lnkeiABCRjFJzvIMnttVz6zsrmVuSGZd8GI4CQEQyyl89XcO0vBw+/+7MueTDcBQAIpIxXjnSyjO7G7n9XYuZPklv9D4aCgARyQjuznd+vpeyaXl8dhLf6H00FAAikhFeeL2Zlw+0cMd7lzA1P6F7YU16CgARmfTcne9s3EvF9Cl88orzwi4nZSgARGTSe+q147xW385/e9/55Odkh11OylAAiMik1j8Q4a+ermHprGnc+Pb5YZeTUhQAIjKpPf5KPQeaTvMn719GdpaFXU5KUQCIyKTV3TfA3zy7j1ULSvnABbPDLiflKABEZNL60aYjNLR18+UPLMNMn/7jJRQAZrbWzGrMrNbM7hxieb6ZPRos32RmlcH868xsq5ntDP5eE/Oay4L5tWb2d6atIyLj6GRnD3/77D6uXlrGVUvKwi4nJY0YAGaWDdwLfBBYCXzSzFbGNbsNaHX3JcA9wN3B/Gbgw+5+EXAr8HDMa74H3A4sDR5rz6EfIiJv8Z2f19DVO8Cffjh+dyWDEvkGsBqodfcD7t4LPAKsi2uzDngomH4MuNbMzN23uXtDMH8XUBB8W5gLFLv7S+7uwA+AG8+5NyIiwLYjrTxafZTb1ixiyayisMtJWYkEwHzgaMzzumDekG3cvR9oA2bGtfkosM3de4L2dSOsEwAzu93Mqs2suqmpKYFyRSSTDUScu/51F7OL8/lChl/ueSSJBMBQY/M+mjZmdgHRYaHPjWKd0Znu97l7lbtXlZdn3k2bRWR0HtlyhJ31bXz1QyuYpks+nFUiAVAHLIh5XgE0DNfGzHKAEqAleF4BPAHc4u77Y9pXjLBOEZFRqT91hr94ci9XLp7BDavmhV1OykskALYAS81skZnlAeuBDXFtNhA9yAtwE/Ccu7uZlQI/A77i7r8ebOzux4AOM7syOPvnFuBfz7EvIpLBIhHnv//zDiLufOejq3TaZwJGDIBgTP8OYCOwB/iJu+8ys2+Z2Q1BsweAmWZWC3wRGDxV9A5gCfB1M9sePGYFyz4P3A/UAvuBp8arUyKSeR566RD/sf8kX79+JefNLAy7nLRg0ZNw0kNVVZVXV1eHXYaIpJjaEx38xt+9yJolZdx/a5U+/ccxs63uXhU/X78EFpG01t03wB89sp3CvGz+/KMXaec/CjpELiJpy9356hM72dXQzv23VDGrqCDsktKKvgGISNr6wUuHefyVev74fUt530pd7G20FAAikpY2H2zhz/59N+9bMYs/vEY/+BoLBYCIpJ0DTZ18/odbOW9GIf/nE5eQpev8j4kCQETSyvG2bj79wGYA7r+1iuKC3JArSl86CCwiaeNUVy+3PLiJtjN9/NPvXsni8mlhl5TW9A1ARNJCe3cfn/nHLRxq7uK+Wy7jooqSsEtKe/oGICIpL/rJfzO7G9r57qcu5Z1v0w1exoMCQERS2snOHn7rgc3sP9HJ3//WZTrdcxwpAEQkZR1t6eIz/7iFutYuHvjtKq5eqkvCjycFgIikpOpDLdz+8Fb6ByI89JnVXLE4/h5Tcq4UACKSch5/pY47/2Un86dP4YFbq3S2zwRRAIhIyujuG+DbP9vND18+wjvfNpP/e/OllBbmhV3WpKUAEJGUcKCpkz/48Tb2HGvnc+9azJc+sIzcbJ2pPpEUACISKnfn0S1H+bN/301eThYP/nYV1yzXmT7JoAAQkdAcb+vmzsdf5fmaJt6xeCZ//fFVzCudEnZZGUMBICJJ1z8Q4Z82H+EvN9bQOxDhmzdcwKevXKiLuiWZAkBEkuql/Sf55r/tYu/xDt6xeCb/+yMXsahsathlZSQFgIgkxat1p7jnmX38oqaJ+aVT+N7Nl7L2wjm6hWOIFAAiMqF2NbRxzzOv8+yeRkoLc/ny2mV89qpFFORmh11axlMAiMiE2N3Qzt88u4+ndzdSVJDDF687n89cVUmRrt+fMhQAIjKuth5u4XvPH+DZPY0U5efwR9cu5bNrFlEyRTv+VJNQAJjZWuBvgWzgfnf/i7jl+cAPgMuAk8An3P2Qmc0EHgMuB/7R3e+Iec3zwFzgTDDr/e5+4ty6IyJh6BuI8OzuRh789UG2HGqltDA3uuO/ahElhdrxp6oRA8DMsoF7geuAOmCLmW1w990xzW4DWt19iZmtB+4GPgF0A18HLgwe8W529+pz7IOIhORQ82l+Un2Un1TX0dzZw/zSKdx1/UrWr15AYZ4GGFJdIltoNVDr7gcAzOwRYB0QGwDrgG8E048B3zUzc/fTwItmtmT8ShaRMDV19PCzVxv46fYGth89RZbBNctn86krFvDu82eRrXP500YiATAfOBrzvA64Yrg27t5vZm3ATKB5hHX/g5kNAP8CfNvdPb6Bmd0O3A5w3nnnJVCuiIwnd+dg82me3dPI07sa2XqkFXdYMbeYr35oOTesms+ckoKwy5QxSCQAhorz+B11Im3i3ezu9WZWRDQAPk30OMJbV+J+H3AfQFVV1UjrFJFx0NXbz6YDLTxfc4Ln9zVx+GQXABfMK+aPrz2ftRfOYdmcopCrlHOVSADUAQtinlcADcO0qTOzHKAEaDnbSt29PvjbYWY/JjrU9F8CQEQmXiTi7D7Wzi/3NfGrfU28cqSVvgGnIDeLd76tjNvWLOK9y2axYEZh2KXKOEokALYAS81sEVAPrAc+FddmA3Ar8BJwE/DcUMM5g4KQKHX3ZjPLBa4Hnh1D/SIyRq2ne/nV6038sqaJX73eRHNnLxD9lP/ZNYu4ekk5VZXT9YOtSWzEAAjG9O8ANhI9DfRBd99lZt8Cqt19A/AA8LCZ1RL95L9+8PVmdggoBvLM7Ebg/cBhYGOw888muvP//rj2TETe4kzvANWHW3j5wEn+Y/9Jdhw9RcRhemEu7z6/nHcvK2fNknLKi/LDLlWSxM7yQT3lVFVVeXW1zhoVGYm7U9d6hurDLWw7coptR06x51g7/REnO8u4uKKEq5eW895l5VxcUaozdyY5M9vq7lXx83Wirsgk0DcQYe+xDl450srWw61sOdTCsbZuAArzsllVUcrvvmsxVyyaQVXlDKbl6399UQCIpJ0zvQPUNHaw51g7uxra2NXQzp5j7XT3RQCYVZTP5YtmsLpyBpdXzmDZnCJ9wpchKQBEUtjJzh52NbQHjzZ2H2vnUPNpIsHIbVF+DivmFfOp1Qt5+3mlXLpwOvNKCnSJZUmIAkAkBfQNRDjYfJp9jR3sa+yMfrqvb6MhGMYBmF86hZXzirn+4nmsnFvEirnFLJheqLtoyZgpAESSrOV0LzXHO9jX2MHuhnZ2HWtj3/FOegeiQzhmsGjmVKoqZ3DR/BIumFfMynnFlBbmhVy5TDYKAJEJ0D8Qof7UGQ6d7OJQ82n2N3Wyr7GD2hOdb5xvD9FTMC+YV8Jnrqpk+dwils4qYsmsaTr3XpJCASAyBt19A9S1nqH+1BkaTp3h2KkzNLR1U996hqOtXRxr62Yg8uYp1kUFOSydNY1rls/i/NlFLJtTxPmzi5hVlK/xegmNAkAkzumefhrbu2ls7+FERzeN7d0cb+vhWFt0Z19/qpvmzp63vMYsevZNxfRCLls4nYrpU1g4YyqVZVOpnFlIuXb0koIUAJIR3J3Wrj6Ot0V36E0dPTSf7uFkZy/NnT00tndzoj3693TvwH95fWFeNvNKpzC3pIAVc4uZXzqFihlTmF9ayLzSAmYXF5CbnRVCz0TGTgEgacnd6ezpp+1MH+1n+jnV1UtLVy+tXX2c7OyhubOH5o5emmJ27oMHWWNNzctm5rR8Zhfns2JeMe9eVs7s4gJmF+czq6jgjelp+Tn6BC+TjgJAxszd6Y84A4MPd/oHnP5IhIFIdLpvIDrd0x+hbyBCb3+E3sG//RF6+iN09w1EH/0RzvRGp7t6o48zff109gxwuqef0z39dHT309HdR2dPP5GzXMVkemEuZdPyKS/K5/LKGcwqzmd2UQFzSqI79VlF+ZRNy2dKng62SubKiAD49AObONh8+o3nZ/sgZzG3NohtF/uS4T4J2rBPRree4V4z3P4u9npOb2njbz53dxxwB8ejf/2tyyIenR/x6LyIOxGPzo9EotMDwfRA0HYi5OVkUZiXzdS8HKbkZTM1L5up+TnMmFpIUUEORfk5FBXkUjwlh5IpuZRMyaW0MI8ZU/MoLcxlemGehmNEEpARAXDR/JI3r3B4lp1W7KLhdqrD7fSGe+1wbUa8Xc4bzd7a0BJIlviQsTemo8vemGfR9ZlBVjCdlfXm8uwsIysIqeg0ZGUZ2WZvLMvOMnKy35yXk2XkZGe9MZ0bTOflZJGXnUVudhb5udHpvJwsCnKzyc/JIj8niyl52RTkZOuHTSJJkhEB8OW1y8MuQUQk5eh7sohIhlIAiIhkKAWAiEiGUgCIiGQoBYCISIZSAIiIZCgFgIhIhlIAiIhkKBvuV6upyMyagMPnsIoyoHmcygnbZOnLZOkHqC+parL05Vz6sdDdy+NnplUAnCszq3b3qrDrGA+TpS+TpR+gvqSqydKXieiHhoBERDKUAkBEJENlWgDcF3YB42iy9GWy9APUl1Q1Wfoy7v3IqGMAIiLypkz7BiAiIgEFgIhIhsqIADCzL5mZm1nZMMsHzGx78NiQ7PpGI4G+3GpmrwePW5Nd30jM7M/M7NXgv/XTZjZvmHYpv01G0ZeU3iYAZvaXZrY36M8TZlY6TLtDZrYz6HN1sutMxCj6stbMasys1szuTHadIzGzj5nZLjOLmNmwp3+e0zZx90n9ABYAG4n+gKxsmDadYdc5Hn0BZgAHgr/Tg+npYdcdV2NxzPQfAn+frtskkb6kwzYJ6nw/kBNM3w3cPUy7Q8P9f5Qqj0T6AmQD+4HFQB6wA1gZdu1xNa4AlgHPA1VnaTfmbZIJ3wDuAb5MwnfhTWkj9eUDwDPu3uLurcAzwNpkFZcId2+PeTqVNN4uCfYl5bcJgLs/7e79wdOXgYow6zkXCfZlNVDr7gfcvRd4BFiXrBoT4e573L1mIt9jUgeAmd0A1Lv7jhGaFphZtZm9bGY3JqO20UqwL/OBozHP64J5KcXM/peZHQVuBu4aplnKbxNIqC9psU3ifBZ4aphlDjxtZlvN7PYk1jRWw/UlHbfLcMa8TdL+pvBm9iwwZ4hFXwO+SvTr4EjOc/cGM1sMPGdmO919/3jWmYhx6IsNMS/pn7DP1g93/1d3/xrwNTP7CnAH8KdDtE35bZJgX1Jim8DIfQnafA3oB340zGquCrbLLOAZM9vr7r+amIqHNw59SYntkkg/EjDmbZL2AeDu7xtqvpldBCwCdpgZRL8GvhfDrEAAAAF7SURBVGJmq939eNw6GoK/B8zseeDtRMcHk2oc+lIHvCfmeQXR8cOkGq4fQ/gx8DOGCIBU3yZDGK4vKbFNYOS+BAeorweu9WBweYh1DG6XE2b2BNGhlKQHwDj0pY7oMbVBFUDD+FWYmFH8+zrbOsa+TcI+0JGsB8McKCF6YC4/mC4DXifFDgaNoi8zgINBn6YH0zPCrjeuxqUx018AHkvXbZJgX1J+mwR1rgV2A+VnaTMVKIqZ/g9gbdi1j7EvOUQPyC/izYPAF4Rd+zC1Ps8wB4HPdZuE3rkk/kd8Y6cJVAH3B9PvBHYG/wB2AreFXetY+xI8/yxQGzw+E3atQ9T+L8BrwKvAvwHz03WbJNKXdNgmQY21RMfEtwePvw/mzwOeDKYXB9tkB7CL6DBF6LWPpS/B8w8B+4h+s0y5vgC/SfSbSg/QCGwc722iS0GIiGSoSX0WkIiIDE8BICKSoRQAIiIZSgEgIpKhFAAiIhlKASAikqEUACIiGeo/ATfE9Ce79NbNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.422274692522746e-05\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>0.452377</td>\n",
       "      <td>0.449104</td>\n",
       "      <td>0.452181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>0.129239</td>\n",
       "      <td>0.125966</td>\n",
       "      <td>0.120726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>RAD_24</td>\n",
       "      <td>0.103221</td>\n",
       "      <td>0.099911</td>\n",
       "      <td>0.099135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>B</td>\n",
       "      <td>0.078899</td>\n",
       "      <td>0.079327</td>\n",
       "      <td>0.077388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>0.059660</td>\n",
       "      <td>0.060075</td>\n",
       "      <td>0.060328</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>RAD_7</td>\n",
       "      <td>0.037664</td>\n",
       "      <td>0.036712</td>\n",
       "      <td>0.041814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>RAD_8</td>\n",
       "      <td>0.030071</td>\n",
       "      <td>0.030423</td>\n",
       "      <td>0.035717</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>RAD_3</td>\n",
       "      <td>0.027465</td>\n",
       "      <td>0.028356</td>\n",
       "      <td>0.036489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>0.013818</td>\n",
       "      <td>0.011491</td>\n",
       "      <td>0.000287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>-0.001228</td>\n",
       "      <td>-0.001077</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>RAD_4</td>\n",
       "      <td>-0.008698</td>\n",
       "      <td>-0.008646</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>RAD_5</td>\n",
       "      <td>-0.011517</td>\n",
       "      <td>-0.011183</td>\n",
       "      <td>-0.002728</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>RAD_2</td>\n",
       "      <td>-0.040970</td>\n",
       "      <td>-0.039042</td>\n",
       "      <td>-0.026297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>RAD_6</td>\n",
       "      <td>-0.055736</td>\n",
       "      <td>-0.055449</td>\n",
       "      <td>-0.046790</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>RAD_1</td>\n",
       "      <td>-0.081501</td>\n",
       "      <td>-0.081083</td>\n",
       "      <td>-0.070050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>TAX</td>\n",
       "      <td>-0.113548</td>\n",
       "      <td>-0.110192</td>\n",
       "      <td>-0.097387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>-0.151553</td>\n",
       "      <td>-0.145910</td>\n",
       "      <td>-0.138506</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>PTRATIO</td>\n",
       "      <td>-0.187898</td>\n",
       "      <td>-0.187369</td>\n",
       "      <td>-0.183970</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>-0.221874</td>\n",
       "      <td>-0.212619</td>\n",
       "      <td>-0.203124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>-0.386249</td>\n",
       "      <td>-0.374867</td>\n",
       "      <td>-0.369251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>-0.476675</td>\n",
       "      <td>-0.474603</td>\n",
       "      <td>-0.477812</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    columns      coef  coef_ridge  coef_lasso\n",
       "5        RM  0.452377    0.449104    0.452181\n",
       "1        ZN  0.129239    0.125966    0.120726\n",
       "20   RAD_24  0.103221    0.099911    0.099135\n",
       "10        B  0.078899    0.079327    0.077388\n",
       "3      CHAS  0.059660    0.060075    0.060328\n",
       "18    RAD_7  0.037664    0.036712    0.041814\n",
       "19    RAD_8  0.030071    0.030423    0.035717\n",
       "14    RAD_3  0.027465    0.028356    0.036489\n",
       "2     INDUS  0.013818    0.011491    0.000287\n",
       "6       AGE -0.001228   -0.001077   -0.000000\n",
       "15    RAD_4 -0.008698   -0.008646   -0.000000\n",
       "16    RAD_5 -0.011517   -0.011183   -0.002728\n",
       "13    RAD_2 -0.040970   -0.039042   -0.026297\n",
       "17    RAD_6 -0.055736   -0.055449   -0.046790\n",
       "12    RAD_1 -0.081501   -0.081083   -0.070050\n",
       "8       TAX -0.113548   -0.110192   -0.097387\n",
       "4       NOX -0.151553   -0.145910   -0.138506\n",
       "9   PTRATIO -0.187898   -0.187369   -0.183970\n",
       "0      CRIM -0.221874   -0.212619   -0.203124\n",
       "7       DIS -0.386249   -0.374867   -0.369251\n",
       "11    LSTAT -0.476675   -0.474603   -0.477812"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_boston\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "\n",
    "\n",
    "df = pd.read_csv('mm_boston_housing.csv')#读取上述处理后的数据\n",
    "\n",
    "y=df['MEDV']\n",
    "X=df.drop(['MEDV','log_MEDV'],axis=1)\n",
    "feat_names = X.columns\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=33,test_size=0.2)#将数据分成训练集和测试集两部分\n",
    "\n",
    "lasso = LassoCV()#Lasso回归\n",
    "lasso.fit(X_train,y_train)\n",
    "\n",
    "\n",
    "y_test_pred_lasso = lasso.predict(X_test)#在测试集上预测\n",
    "y_train_pred_lasso = lasso.predict(X_train)#在训练集上预测\n",
    "\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef\":list(lasso.coef_.T)})\n",
    "fs.sort_values(by=['coef'],ascending=False)\n",
    "fs.head()\n",
    "print ('The r2 score of lasso on test is', r2_score(y_test,y_test_pred_lasso))#模型训练结果评分\n",
    "print ('The r2 score of lasso on train is', r2_score(y_train,y_train_pred_lasso))\n",
    "\n",
    "\n",
    "mse_mean = np.mean(lasso.mse_path_,axis=1)\n",
    "plt.plot(np.log10(lasso.alphas_),mse_mean)\n",
    "plt.show()\n",
    "\n",
    "\n",
    "print(lasso.alpha_)\n",
    "\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef\":list(lr.coef_.T),\"coef_ridge\":list(ridge.coef_.T),\"coef_lasso\":list(lasso.coef_.T)})\n",
    "fs.sort_values(by=['coef'],ascending=False)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.代码中给出了岭回归（RidgeCV）和Lasso（LassoCV）超参数（alpha_）调优的过程，请结合两个最佳模型以及最小二乘线性回归模型的结果，给出什么场合应该用岭回归，什么场合用Lasso，什么场合用最小二乘"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "答：由上题计算结果，可得如下：\n",
    "- 最小二乘线性回归模型的回归结果评价为：\n",
    "The r2 score of LinearRegression on test is 0.6939789810509469\n",
    "The r2 score of LinearRegression on train is 0.7549146436868177\n",
    "根据最小二乘线性回归的解$(X^TX)$<sup>−1</sup>$X^TY$，需要对矩阵XTX求逆。当输入特征存在共线性时，矩阵X可能不满秩，导致$X^TX$接近奇异，求逆不稳定，结果不理想。另外，由于没有加正则项对各特征系数进行限制，容易导致曲线剧烈震荡。可能在训练集上拟合比较好，而对测试集偏差比较大。<br />\n",
    "- 岭回归模型的回归结果评价为：\n",
    "The r2 score of RidgeCV on test is 0.6961684813070569\n",
    "The r2 score of RidgeCV on train is 0.7548524440445561\n",
    "从结果分析，岭回归在训练集上的得分比最小二乘差一点，而在训练集上的得分比最小二乘略好点。由于岭回归加入正则项，通过alpha超参数调节，综合考虑了拟合程度和模型的复杂性，通过正则项，保证了模型曲线不会剧烈震荡。从岭回归的解为$(X^TX+λI)$<sup>−1</sup>$X^TY$，由于引入了λI，保证了$(X^TX+λI)$求逆可得到稳定解。从而岭回归相对于最小二乘来讲，稳定性更好。\n",
    "- Lasso回归模型的回归结果评价为：\n",
    "The r2 score of lasso on test is 0.6953719445869475\n",
    "The r2 score of lasso on train is 0.7546505272258799\n",
    "从结果分析，Lasso回归模型在训练集上不及最小二乘，但在测试集上效果较好些，但在比不上岭回归。Lasso回归，同样引入正则项，来平衡复杂度和拟合程度。当正则参数取合适值，L1正则能使有些线性回归系数为0。因此，当输入特征多时，L1正则可以分析出特征与目标变量相关性的强弱，去掉较弱的特征项，简化模型。\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:tf1.14]",
   "language": "python",
   "name": "conda-env-tf1.14-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
